Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.
Python agent-orchestrationaiai-agentschatopsdiscordlangchainmcpmicrosoft-teamsmulti-agentopen-sourceopenaiself-hostedslack
E2E harness foundation: fast and provider lanes in one package
Petr Bauch128540862dee
diff --git a/console/tooling-e2e/.env.example b/console/tooling-e2e/.env.exampleindex b64d7c188..96b3a7008 100644--- a/console/tooling-e2e/.env.example+++ b/console/tooling-e2e/.env.example@@ -1,13 +1,21 @@# Copy to .env (gitignored) and fill in. Never commit real values.-SWITCH_API_URL=http://127.0.0.1:54111+# `just e2e-fast` needs none of this: it sets everything for its own stack.++# fast | provider. Unset, every end-to-end scenario skips.+SWITCH_E2E_LANE=fast+# ephemeral | local | dev | staging | prod. prod is read-only for the harness.+SWITCH_E2E_ENVIRONMENT=local++SWITCH_API_URL=http://127.0.0.1:8000SWITCH_GATEWAY_ADMIN_PASSWORD=SWITCH_AGENT_REGISTRATION_TOKEN=-MATTERMOST_URL=http://127.0.0.1:54112+# All three or none. Without them, scenarios that need Mattermost skip.+MATTERMOST_URL=http://127.0.0.1:8065MATTERMOST_TOKEN=MATTERMOST_TEAM=switch-# Optional. Working directory of the session under test — the harness writes the-# agent's credentials to <dir>/.switch/agents/<agent-name>.json when set.+# Provider lane only. Working directory of the session under test — the harness+# writes the agent's credentials to <dir>/.switch/agents/<agent-name>.json when set.# SWITCH_E2E_AGENT_DIR=diff --git a/console/tooling-e2e/README.md b/console/tooling-e2e/README.mdindex 77dd3fcde..069675ac5 100644--- a/console/tooling-e2e/README.md+++ b/console/tooling-e2e/README.md@@ -1,80 +1,167 @@# `@switch-console/tooling-e2e`-An end-to-end harness that sends **real messages through a real Switch server and-a real Mattermost bridge** to exercise Switch Console's provider-backed sessions.--It plays the human: it registers a throwaway agent, creates a room (and with it a-Mattermost channel), posts `@agent …` into that channel, and waits for the-agent's bot to post back. Nothing here talks to Switch Console directly — if a-scenario passes, an operator doing the same thing by hand would have seen the-same result.--Every artefact it creates is prefixed `e2e-` and removed on teardown.+End-to-end tests for Switch: **real messages through a real Switch server**, and+through a real Mattermost bridge where a scenario crosses one. One package, two+runner modes:++- **The fast lane** needs no provider, no Switch Console and no LLM key. The+ harness plays every participant over plain HTTP: it registers throwaway+ agents, creates rooms, posts as agents and as a Mattermost human, and asserts+ on what arrives. It carries no secrets a fork lacks, so it can gate pull+ requests.+- **The provider lane** needs a real agent session (Switch Console or its+ sidecar) attending the room, and checks what that session does: answering,+ asking a question, asking for approval, being interrupted.++Nothing here talks to Switch Console directly. If a scenario passes, an operator+doing the same thing by hand would have seen the same result.## Running it+The fast lane against a throwaway stack built from this checkout (fresh+Postgres and Mattermost, migrations, switch-core on the host), torn down+afterwards. Needs Docker, `uv`, `jq`, and your local stack stopped:+```bash-# from console/tooling-e2e-SWITCH_E2E=1 ../node_modules/.bin/vitest run # everything-SWITCH_E2E=1 ../node_modules/.bin/vitest run src/loopback.integration.test.ts-SWITCH_E2E=1 ../node_modules/.bin/vitest run src/run.integration.test.ts+just e2e-fast # from the repo root+just e2e-fast --keep # leave the stack running afterwards+just e2e-fast -- -t mattermost # arguments after -- go to vitest```-This package is not yet listed in `console/pnpm-workspace.yaml`, so it has no-`node_modules` of its own and runs against the workspace root's hoisted `vitest`-and `typescript`. Adding `- 'tooling-e2e'` to that file's `packages:` list makes-the usual forms work instead:+Its log, JUnit report and the stack's per-run secrets go to `build/e2e/ephemeral/`.+The repository's own `.env` is never read or written.++Against a stack you already have, set the configuration below and run vitest+from this directory:```bash-pnpm --filter @switch-console/tooling-e2e test-pnpm --filter @switch-console/tooling-e2e cleanup+SWITCH_E2E_LANE=fast pnpm test+SWITCH_E2E_LANE=provider pnpm test```-Housekeeping after an interrupted run — deletes every `e2e-*` agent and room on+With `SWITCH_E2E_LANE` unset every end-to-end scenario is skipped, which is+what `pnpm -r test` and PR CI see. The unit tests (configuration, the read-only+guard, naming, scenario headers) run either way.++Housekeeping after an interrupted run deletes every `e2e-*` agent and room onthe Switch server and archives every `e2e-*` Mattermost channel, and nothing-else:+else. It is also what a scheduled wipe of a deployed environment runs:```bash-SWITCH_E2E=1 node --experimental-strip-types src/cleanup.ts+SWITCH_E2E_LANE=fast pnpm cleanup+```++Typecheck: `pnpm typecheck`.++## Layout++| Path | What it holds |+| --- | --- |+| `src/core/` | Configuration, run naming, the read-only guard, scenario headers and admission, lane resolution |+| `src/clients/` | The Switch and Mattermost HTTP clients |+| `src/fixtures/` | Throwaway agents and rooms for a scenario, the bridge-readiness probe, teardown |+| `src/fast/` | Fast-lane scenarios, one file per area, listed in `index.ts` |+| `src/provider/` | The provider lane: Console credential files, the shared agent and channel, the scenarios, `seed.ts` |+| `scripts/ephemeral-stack.sh` | The throwaway-stack runner behind `just e2e-fast` |++## Scenarios and their headers++Every scenario, in either lane, carries a header:++```ts+{+ id: 'addressed-delivery', // kebab-case, unique across both lanes+ covers: ['POST /agents/{agent_id}/message', 'list_rooms'],+ needsProvider: false, // true puts it in the provider lane+ environments: ['ephemeral', 'local', 'dev', 'staging'],+ owner: 'platform', // a team or role, never a person+ requires: ['mattermost'], // what must be configured+}```-Typecheck: `../node_modules/.bin/tsc --noEmit -p tsconfig.json`.+`covers` names agent operations from the registry+(`core/switch_core/bridges/agent/operations/definitions.py`) and HTTP routes+as `METHOD /path/{param}`, with the server's own parameter names. It is what a+coverage gate reads, without running anything. Headers are validated by a unit+test, so a malformed one fails `pnpm test`.++A scenario runs when its lane is the one asked for, the target environment is+in `environments`, and its `requires` are configured. Otherwise it shows as+**skipped, with the reason**, never as passed. Under `SWITCH_E2E_STRICT=1` (how+`just e2e-fast` and CI run) a missing prerequisite, missing configuration or an+unreachable server **fails** the run instead, so a job that ran nothing cannot+go green.++A scenario that changes server-global state (a feature flag, say) lists only+`ephemeral`. One that writes anything never lists `prod`.++### Fast lane++Plain code. A scenario gets fixtures that create agents and rooms under names+unique to the run and the scenario, and removes them afterwards whether it+passed or not. A teardown that leaves something behind fails the scenario.++| Scenario | What it drives | Needs |+| --- | --- | --- |+| `addressed-delivery` | One agent addresses another in a room with no bridge; the recipient's notification carries the room, the sender and the addressed flag. | — |+| `unaddressed-history` | Five unaddressed messages arrive in the recipient's room history in the order sent, and none of them as a notification. | — |+| `operations-front-door` | The operations registry over HTTP: it lists the operations, `list_rooms` returns the room, `read_context` returns a message posted in it. | — |+| `mattermost-roundtrip` | A room on the Mattermost bridge: an addressed post from the human reaches the agent, and the agent's reply appears as its bot. | Mattermost |++If `mattermost-roundtrip` passes and a provider-lane scenario fails, the fault+is in the agent session, not in Switch, the bridge or the harness.++## Naming, isolation and the read-only guard++Everything the harness creates is named `e2e-<runId><scope>-<role>`: the run,+a letter per scenario, then what the thing is for (`e2e-k3x9q2a-snd`). Names go+through one helper, so no scenario builds a raw name, and the helper refuses a+name too long for a Mattermost bot username (22 characters) rather than letting+bot creation fail quietly on the bridge.++The harness's agents are created owner-only, like every agent, and the+fixtures open their addressing through the gateway: their senders are other+throwaway agents and an unclaimed Mattermost user, whose messages would+otherwise be refused rather than delivered.++With `SWITCH_E2E_ENVIRONMENT=prod` the clients refuse every request but reads+and logging in before it leaves the process, so a wrong base URL fails a test+rather than writing to production.## ConfigurationValues come from the process environment, falling back to a **gitignored `.env`**-beside this file. Never commit one.+beside this file. Never commit one. `just e2e-fast` sets all of them itself.| Variable | Required | Meaning || --- | --- | --- |-| `SWITCH_E2E` | yes | Must be `1`. Without it the whole suite skips. |+| `SWITCH_E2E_LANE` | to run anything | `fast` or `provider`. Unset, every end-to-end scenario skips. |+| `SWITCH_E2E_ENVIRONMENT` | yes | `ephemeral`, `local`, `dev`, `staging` or `prod`. Decides which scenarios run, and makes `prod` read-only. || `SWITCH_API_URL` | yes | Switch origin — serves both `/agents/…` and `/gateway/…`. || `SWITCH_GATEWAY_ADMIN_EMAIL` | yes | Gateway admin account. || `SWITCH_GATEWAY_ADMIN_PASSWORD` | yes | Its password. |-| `SWITCH_AGENT_REGISTRATION_TOKEN` | yes | Registration API key; mints the throwaway agent. |-| `MATTERMOST_URL` | yes | Mattermost origin. |-| `MATTERMOST_TOKEN` | yes | Personal access token of the **human** the harness posts as. |-| `MATTERMOST_TEAM` | yes | Team **slug** (e.g. `switch`), not its display name. |-| `SWITCH_E2E_AGENT_DIR` | no | Working directory of the session under test. When set, the agent's credentials are written into it (see below) and it is sent to Switch as the agent's `repo_dir`. |-| `SWITCH_E2E_KEEP` | no | `1` leaves the agent, room and channel behind for inspection. |-| `SWITCH_E2E_MANIFEST` | no | File the setup records its agent and room in, so a second process can reuse them instead of registering its own. See "Seeding first" below. |-| `SWITCH_E2E_REPLY_TIMEOUT_MS` | no | How long a scenario waits for the agent (default 5 min). |-| `SWITCH_E2E_SESSION_TIMEOUT_MS` | no | How long setup waits for a live session (default 2 min). |-| `SWITCH_E2E_AGENT_TYPE` | no | `opencode` (default), `claude-code`, or `codex` — which provider's runtime is under test. Decides the agent name prefix, the Switch known-agent type, and (for `claude-code`) the extra files written into the working dir. |-| `SWITCH_E2E_QUESTION_MODE` | no | `numbered` (default, every agent type) or `prose`; see the `question` row below. |--The suite **skips, with the reason printed**, when `SWITCH_E2E` is unset, when-configuration is missing (it names every missing variable at once), or when-Switch or Mattermost is unreachable. A skip always means "this machine is not set-up", never "the thing under test is broken".+| `SWITCH_AGENT_REGISTRATION_TOKEN` | yes | Registration API key; mints the throwaway agents. |+| `MATTERMOST_URL` | for Mattermost scenarios | Mattermost origin. |+| `MATTERMOST_TOKEN` | for Mattermost scenarios | Token of the **human** the harness posts as (a personal access token, or a login session token). |+| `MATTERMOST_TEAM` | for Mattermost scenarios | Team **slug** (e.g. `switch`), not its display name. |+| `SWITCH_E2E_STRICT` | no | `1` turns every "cannot run" into a failure. CI sets it. |+| `SWITCH_E2E_KEEP` | no | `1` leaves agents, rooms and channels behind for inspection. |+| `SWITCH_E2E_AGENT_DIR` | provider lane | Working directory of the session under test. When set, the agent's credentials are written into it (see below) and it is sent to Switch as the agent's `repo_dir`. |+| `SWITCH_E2E_MANIFEST` | provider lane | File the setup records its agent and room in, so a second process can reuse them. See "Seeding first". |+| `SWITCH_E2E_REPLY_TIMEOUT_MS` | provider lane | How long a scenario waits for the agent (default 5 min). |+| `SWITCH_E2E_SESSION_TIMEOUT_MS` | provider lane | How long setup waits for a live session (default 2 min). |+| `SWITCH_E2E_AGENT_TYPE` | provider lane | `opencode` (default), `claude-code`, `codex`, `antigravity` or `cursor` — which provider's runtime is under test. |+| `SWITCH_E2E_QUESTION_MODE` | provider lane | `numbered` (default) or `prose`; see the `question` row below. |++Mattermost is all or nothing: none of its three variables means "no Mattermost+here"; some of them is reported as a mistake.++# The provider lane## What must be running-`loopback.integration.test.ts` needs nothing but the stack: it plays both the-human and the agent, and proves the harness's own plumbing.--`run.integration.test.ts` needs a **Switch Console session for the agent it just-registered**. Setup polls the agent's status in the room and refuses to run the+The provider lane needs Mattermost and a **Switch Console session for the agent+it just registered**. Setup polls the agent's status in the room and refuses to run thescenarios until Switch reports it `live` (a session is attending the room) or`dormant` (an auto-session watcher is running and will spawn one on demand) —otherwise you get four identical timeouts instead of one clear message.@@ -116,19 +203,19 @@ database, and it reads that database at launch. So the agent has to exist beforethe console starts, and the console has to be up before the first scenarioposts — three steps, in three processes.-`src/seed.ts` is the first of them. It does exactly what `beforeAll` does —+`src/provider/seed.ts` is the first of them. It does exactly what `beforeAll` does —register the agent, create the room and channel, write the credentials — andrecords the result in `SWITCH_E2E_MANIFEST`, above all the agent's API key,which registration returns exactly once:```bash-SWITCH_E2E=1 SWITCH_E2E_MANIFEST=/tmp/e2e.json SWITCH_E2E_AGENT_DIR=/tmp/e2e-work \- node --experimental-strip-types src/seed.ts+SWITCH_E2E_LANE=provider SWITCH_E2E_MANIFEST=/tmp/e2e.json SWITCH_E2E_AGENT_DIR=/tmp/e2e-work \+ node --experimental-strip-types src/provider/seed.ts# add the agent to the console's database, pointed at that working directory,# with provider_config {"version":"2","providerId":"opencode","values":{}};# start Switch Console-SWITCH_E2E=1 SWITCH_E2E_MANIFEST=/tmp/e2e.json SWITCH_E2E_AGENT_DIR=/tmp/e2e-work \- ../node_modules/.bin/vitest run src/run.integration.test.ts+SWITCH_E2E_LANE=provider SWITCH_E2E_MANIFEST=/tmp/e2e.json SWITCH_E2E_AGENT_DIR=/tmp/e2e-work \+ pnpm test src/provider```For Claude Code, add `SWITCH_E2E_AGENT_TYPE=claude-code` to both commands and@@ -147,7 +234,7 @@ the room before the greeting proceeds.Setup finds the manifest, reuses what is in it rather than registering a secondagent, and deletes it during teardown — so a manifest never outlives its room.-## The scenarios+## The provider scenarios| Scenario | What it drives | Passes when || --- | --- | --- |@@ -202,6 +289,8 @@ POST /gateway/rooms {name, description, bridge_id, agent_nMattermost channel and adds the botDELETE /gateway/rooms/{id}DELETE /gateway/agents/by-name/{name} (admin teardown, no agent key needed)+PUT /gateway/agents/{id}/addressing-policy {"policy": null} -> anyone may address it+POST /gateway/rooms with internal_only: true and no bridge_id -> a room with no channel```### Switch agent bridge@@ -222,6 +311,10 @@ GET /agents/{id}/rooms/{room}/history?limit= -> {"events":[{sender,sender_nPOST /agents/{id}/message {"room_id":"…","content":"…"}-> {"ok":true,"event_id":"$…"}POST /agents/{id}/watch/heartbeat the auto_session "I am watching" beat+GET /agents/{id}/ops -> {"operations": {<name>: {…}, …}} the registry, keyed by name+POST /agents/{id}/ops/{operation} body = the operation's arguments+ -> {"result": …}. Operations that act on "the room you are connected to"+ need a live connection; those taking the room as an argument do not.DELETE /agents/{id} agent's own key only, not the registration token```@@ -259,8 +352,8 @@ GET /api/v4/channels/{id}/posts?since=<ms> -> {order:[…newest first…],`!interrupt` addresses nobody and the admin bot says so. `!interrupt-all-agents`is the room-wide form. The same shape applies to `!reset` and `!compact`.- Agent names must be valid Mattermost usernames — lowercase, 3–22 characters,- `[a-z0-9._-]` — because they become bot usernames. `e2e-opencode-` plus a- 6-character run id fits.+ `[a-z0-9._-]` — because they become bot usernames. The naming helper+ (`src/core/run.ts`) enforces it.## Two traps this harness works around@@ -287,7 +380,7 @@ turn.Switch posts its "I'm not online in this room" onboarding notice **as the agent'sown bot account**, not as the Switch Admin bot. Any scenario that accepts "somepost from the bot" therefore passes with no session running at all. Those phrases-are listed in `NO_SESSION_MARKERS` (`src/scenarios.ts`), never count as a match,+are listed in `NO_SESSION_MARKERS` (`src/provider/scenarios.ts`), never count as a match,and are reported as "no live Switch Console session" rather than as a timeout.Keep that list in step with the wording in`core/switch_core/gateway/known_agents.py` anddiff --git a/console/tooling-e2e/package.json b/console/tooling-e2e/package.jsonindex 0ed977d9f..c327f7f8b 100644--- a/console/tooling-e2e/package.json+++ b/console/tooling-e2e/package.json@@ -3,7 +3,7 @@"version": "0.0.0","private": true,"type": "module",- "description": "End-to-end harness that drives real messages through a Switch server and a Mattermost bridge to exercise Switch Console's provider-backed sessions.",+ "description": "End-to-end harness for Switch: a provider-free fast lane over plain HTTP, and a provider lane that drives real agent sessions through Switch Console.","scripts": {"test": "vitest run","test:e2e": "vitest run",diff --git a/console/tooling-e2e/scripts/ephemeral-stack.sh b/console/tooling-e2e/scripts/ephemeral-stack.shnew file mode 100755index 000000000..384d1e75a--- /dev/null+++ b/console/tooling-e2e/scripts/ephemeral-stack.sh@@ -0,0 +1,130 @@+#!/usr/bin/env bash+# Run the harness against a fresh, throwaway Switch stack built from this+# checkout: new Postgres and Mattermost volumes, migrations, switch-core on the+# host, then the chosen lane, then teardown.+#+# just e2e-fast the fast lane (no provider, no Console)+# just e2e-fast --keep leave the stack running afterwards+# just e2e-fast -- <vitest args> pass arguments through to vitest+#+# Secrets are generated per run into a temporary env file. The repository's+# own .env is never read or written, so this cannot rotate a developer's live+# secrets or pick up a stale value from them.+#+# Ports 5432, 8000, 8065 and 8081 must be free: stop a local stack first.+# Everything the run writes goes to build/e2e/ephemeral/ (or $E2E_OUT).+set -euo pipefail++REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." && pwd)"+HARNESS="$REPO/console/tooling-e2e"+OUT="${E2E_OUT:-$REPO/build/e2e/ephemeral}"+LANE="${SWITCH_E2E_LANE:-fast}"+KEEP=0+VITEST_ARGS=()+while [ $# -gt 0 ]; do+ case "$1" in+ --keep) KEEP=1 ;;+ --) shift; VITEST_ARGS=("$@"); break ;;+ *) echo "unknown option: $1" >&2; exit 2 ;;+ esac+ shift+done++for tool in docker uv openssl curl jq; do+ command -v "$tool" >/dev/null || { echo "$tool is required but not on PATH." >&2; exit 1; }+done+for port in 5432 8000 8065 8081; do+ if lsof -nP -iTCP:"$port" -sTCP:LISTEN >/dev/null 2>&1; then+ echo "Port $port is in use. Stop your local stack first ('just down', and any 'just run')." >&2+ exit 1+ fi+done++rm -rf "$OUT"; mkdir -p "$OUT"+ENV_FILE="$OUT/stack.env"+cp "$REPO/.env.example" "$ENV_FILE"+for key in DB_PASSWORD DB_OWNER_PASSWORD AGENT_REGISTRATION_TOKEN GATEWAY_ADMIN_PASSWORD \+ MATTERMOST_ADMIN_PASSWORD MATTERMOST_USER_PASSWORD; do+ sed -i.bak "s|^${key}=.*|${key}=$(openssl rand -hex 24)|" "$ENV_FILE"+done+sed -i.bak "s|^SECRET_KEYS=.*|SECRET_KEYS=e2e:$(openssl rand -hex 32)|" "$ENV_FILE"+rm -f "$ENV_FILE.bak"+chmod 600 "$ENV_FILE"+set -a; source "$ENV_FILE"; set +a++cd "$REPO"+COMPOSE=(docker compose -p switch-e2e-ephemeral -f deploy/local/docker-compose.yml+ --env-file "$ENV_FILE" --project-directory .)+SERVER_PID=""+cleanup() {+ local status=$?+ if [ "$KEEP" = 0 ]; then+ [ -n "$SERVER_PID" ] && kill "$SERVER_PID" 2>/dev/null && wait "$SERVER_PID" 2>/dev/null+ "${COMPOSE[@]}" down -v >/dev/null 2>&1 || true+ rm -f "$ENV_FILE"+ else+ echo "Left running: switch-core :8000 (pid $SERVER_PID), Mattermost :8065."+ echo "Secrets for this stack: $ENV_FILE. Tear down with:"+ echo " kill $SERVER_PID; ${COMPOSE[*]} down -v"+ fi+ exit "$status"+}+trap cleanup EXIT++echo "Starting a fresh stack (Postgres, Mattermost)…"+"${COMPOSE[@]}" down -v >/dev/null 2>&1 || true+"${COMPOSE[@]}" up -d --build >"$OUT/compose-up.log" 2>&1 \+ || { echo "docker compose up failed; see $OUT/compose-up.log" >&2; exit 1; }++for _ in $(seq 60); do+ "${COMPOSE[@]}" exec -T postgres pg_isready -q >/dev/null 2>&1 && break; sleep 1+done+uv run --project core alembic -c core/alembic.ini upgrade head >"$OUT/migrate.log" 2>&1 \+ || { echo "migrations failed; see $OUT/migrate.log" >&2; exit 1; }++echo "Starting switch-core from this checkout…"+GATEWAY_COOKIE_SECURE=false uv run --project core python -m switch_core.main \+ >"$OUT/switch-core.log" 2>&1 &+SERVER_PID=$!+for _ in $(seq 120); do curl -sf localhost:8000/health >/dev/null && break; sleep 1; done+curl -sf localhost:8000/health >/dev/null \+ || { echo "switch-core did not come up; see $OUT/switch-core.log" >&2; exit 1; }++echo "Waiting for the Mattermost bridge to be registered…"+for _ in $(seq 150); do+ "${COMPOSE[@]}" logs setup 2>&1 | grep -q "Setup complete" && break; sleep 2+done+"${COMPOSE[@]}" logs setup 2>&1 | grep -q "Setup complete" \+ || { "${COMPOSE[@]}" logs setup >"$OUT/setup.log" 2>&1+ echo "Mattermost setup never completed; see $OUT/setup.log" >&2; exit 1; }++# The harness posts as the Mattermost human the setup container created. A+# session token from logging in authenticates exactly as a personal access+# token does, and dies with the stack.+MATTERMOST_TOKEN="$(curl -sf -D - -o /dev/null -H 'Content-Type: application/json' \+ -d "$(jq -n --arg u "$MATTERMOST_USER" --arg p "$MATTERMOST_USER_PASSWORD" '{login_id: $u, password: $p}')" \+ http://localhost:8065/api/v4/users/login | tr -d '\r' | awk -F': ' 'tolower($1)=="token" {print $2}')"+[ -n "$MATTERMOST_TOKEN" ] || { echo "could not log in to Mattermost as $MATTERMOST_USER" >&2; exit 1; }++echo "Running the $LANE lane…"+status=0+(+ cd "$HARNESS"+ SWITCH_E2E_LANE="$LANE" \+ SWITCH_E2E_ENVIRONMENT=ephemeral \+ SWITCH_E2E_STRICT=1 \+ SWITCH_API_URL=http://localhost:8000 \+ SWITCH_GATEWAY_ADMIN_EMAIL="$GATEWAY_ADMIN_EMAIL" \+ SWITCH_GATEWAY_ADMIN_PASSWORD="$GATEWAY_ADMIN_PASSWORD" \+ SWITCH_AGENT_REGISTRATION_TOKEN="$AGENT_REGISTRATION_TOKEN" \+ MATTERMOST_URL=http://localhost:8065 \+ MATTERMOST_TOKEN="$MATTERMOST_TOKEN" \+ MATTERMOST_TEAM="$MATTERMOST_TEAM_NAME" \+ pnpm exec vitest run --reporter=verbose --reporter=junit \+ --outputFile.junit="$OUT/junit.xml" ${VITEST_ARGS[@]+"${VITEST_ARGS[@]}"}+) || status=$?++if [ "$status" != 0 ]; then+ echo "The $LANE lane failed. switch-core log: $OUT/switch-core.log" >&2+fi+exit "$status"diff --git a/console/tooling-e2e/src/cleanup.ts b/console/tooling-e2e/src/cleanup.tsindex d574f8105..0948d054b 100644--- a/console/tooling-e2e/src/cleanup.ts+++ b/console/tooling-e2e/src/cleanup.ts@@ -1,31 +1,23 @@/**- * Remove artefacts a crashed or interrupted run left behind: Switch agents named- * `e2e-*`, Switch rooms named `e2e-*`, and Mattermost channels named `e2e-*`.+ * Remove artefacts a crashed or interrupted run left behind: Switch agents,+ * Switch rooms and Mattermost channels whose names start with `e2e-`.*- * Scoped strictly to the `e2e-` prefix — an operator's real agents (`switchdev`- * and friends) and their rooms are never in scope, and the script prints- * everything it is about to touch.+ * Scoped strictly to that prefix — an operator's real agents and their rooms are+ * never in scope, and the script prints everything it touches. It is also what+ * a scheduled wipe of a deployed environment runs.*- * SWITCH_E2E=1 node --experimental-strip-types src/cleanup.ts+ * SWITCH_E2E_LANE=fast node --experimental-strip-types src/cleanup.ts*/-import { loadEnv } from './env.ts';-import { MattermostClient } from './mattermost-client.ts';-import { SwitchClient } from './switch-client.ts';--const PREFIX = 'e2e-';+import { ARTIFACT_PREFIX } from './core/run.ts';+import { message, resolveLane } from './core/suite.ts';export async function cleanup(log: (message: string) => void = console.log): Promise<void> {- const env = loadEnv();- const switchClient = new SwitchClient({- apiUrl: env.switchApiUrl,- registrationToken: env.agentRegistrationToken,- gatewayAdminEmail: env.gatewayAdminEmail,- gatewayAdminPassword: env.gatewayAdminPassword,- });- const mattermost = new MattermostClient({ url: env.mattermostUrl, token: env.mattermostToken });+ const lane = await resolveLane('fast');+ if (!lane.target) throw new Error(`Cannot clean up: ${lane.skip}`);+ const { env, switchClient, mattermost } = lane.target;for (const room of await switchClient.listRooms()) {- if (!room.name.startsWith(PREFIX)) continue;+ if (!room.name.startsWith(ARTIFACT_PREFIX)) continue;try {await switchClient.deleteRoom(room.id);log(`deleted room ${room.name} (${room.id})`);@@ -35,7 +27,7 @@ export async function cleanup(log: (message: string) => void = console.log): Pro}for (const agent of await switchClient.listAgents()) {- if (!agent.name.startsWith(PREFIX)) continue;+ if (!agent.name.startsWith(ARTIFACT_PREFIX)) continue;try {await switchClient.deleteAgentByName(agent.name);log(`deleted agent ${agent.name} (${agent.id})`);@@ -44,9 +36,13 @@ export async function cleanup(log: (message: string) => void = console.log): Pro}}- const team = await mattermost.findTeam(env.mattermostTeam);+ if (!mattermost || !env.mattermost) {+ log('no Mattermost configured — skipping channels');+ return;+ }+ const team = await mattermost.findTeam(env.mattermost.team);for (const channel of await mattermost.listTeamChannels(team.id)) {- if (!channel.name.startsWith(PREFIX)) continue;+ if (!channel.name.startsWith(ARTIFACT_PREFIX)) continue;try {await mattermost.archiveChannel(channel.id);log(`archived channel ${channel.name} (${channel.id})`);@@ -56,10 +52,6 @@ export async function cleanup(log: (message: string) => void = console.log): Pro}}-function message(error: unknown): string {- return error instanceof Error ? error.message : String(error);-}-if (import.meta.url === `file://${process.argv[1]}`) {cleanup().catch((error: unknown) => {console.error(message(error));diff --git a/console/tooling-e2e/src/mattermost-client.ts b/console/tooling-e2e/src/clients/mattermost-client.tssimilarity index 93%rename from console/tooling-e2e/src/mattermost-client.tsrename to console/tooling-e2e/src/clients/mattermost-client.tsindex 4db7ef0d9..a2c31f788 100644--- a/console/tooling-e2e/src/mattermost-client.ts+++ b/console/tooling-e2e/src/clients/mattermost-client.ts@@ -1,3 +1,6 @@+import type { Environment, MattermostEnv } from '../core/env.ts';+import { assertAllowed } from '../core/guard.ts';+/*** Mattermost REST client for the harness — the *human* half of the loop.*@@ -56,18 +59,15 @@ export class MattermostHttpError extends Error {}}-/** Mattermost usernames: lowercase, 3-22 chars, `[a-z0-9._-]`, must start alphanumeric. */-export function isValidMattermostUsername(name: string): boolean {- return /^[a-z0-9][a-z0-9._-]{2,21}$/.test(name);-}-export class MattermostClient {private readonly baseUrl: string;private readonly token: string;+ private readonly environment: Environment;- constructor(params: { url: string; token: string }) {- this.baseUrl = params.url.endsWith('/') ? params.url.slice(0, -1) : params.url;- this.token = params.token;+ constructor(env: MattermostEnv, environment: Environment) {+ this.baseUrl = env.url.endsWith('/') ? env.url.slice(0, -1) : env.url;+ this.token = env.token;+ this.environment = environment;}private async request<T>(@@ -76,6 +76,7 @@ export class MattermostClient {body?: unknown,options: { allow404?: boolean } = {}): Promise<T> {+ assertAllowed(this.environment, method, `/api/v4${path}`);const headers: Record<string, string> = {Authorization: `Bearer ${this.token}`,Accept: 'application/json',@@ -137,10 +138,7 @@ export class MattermostClient {/** `GET /api/v4/teams/{team_id}/channels` — the team's public channels. */async listTeamChannels(teamId: string): Promise<MattermostChannel[]> {- return this.request<MattermostChannel[]>(- 'GET',- `/teams/${teamId}/channels?per_page=200`- );+ return this.request<MattermostChannel[]>('GET', `/teams/${teamId}/channels?per_page=200`);}/** `GET /api/v4/channels/{id}`. */@@ -267,11 +265,7 @@ export class MattermostClient {* scenario, which asserts the *absence* of further output rather than the* presence of a reply.*/- async postsFrom(- channelId: string,- userId: string,- sinceMs: number- ): Promise<MattermostPost[]> {+ async postsFrom(channelId: string, userId: string, sinceMs: number): Promise<MattermostPost[]> {const posts = await this.postsSince(channelId, sinceMs);return posts.filter((post) => post.user_id === userId && !post.type?.startsWith('system_'));}diff --git a/console/tooling-e2e/src/switch-client.ts b/console/tooling-e2e/src/clients/switch-client.tssimilarity index 83%rename from console/tooling-e2e/src/switch-client.tsrename to console/tooling-e2e/src/clients/switch-client.tsindex bf4ddb12e..1b56c7709 100644--- a/console/tooling-e2e/src/switch-client.ts+++ b/console/tooling-e2e/src/clients/switch-client.ts@@ -1,3 +1,6 @@+import type { Environment, SwitchEnv } from '../core/env.ts';+import { assertAllowed } from '../core/guard.ts';+/*** Thin client for the two Switch HTTP surfaces the harness talks to, both served* by the same origin (`SWITCH_API_URL`):@@ -5,9 +8,11 @@* - the **agent bridge** at `/agents/...`, authenticated with either the* registration token (to mint an agent) or an agent's own API key (to act as* that agent);- * - the **gateway** at `/gateway/...`, authenticated with an admin bearer token+ * - the **gateway** at `/gateway/...`, authenticated with the session cookie* from `POST /gateway/auth/login`.*+ * Every request passes the read-only guard first (`core/guard.ts`).+ ** Only endpoints this harness has actually exercised against a live server are* modelled here; see README.md for the verified request/response shapes.*/@@ -97,11 +102,12 @@ function truncate(text: string, max: number): string {return text.length > max ? `${text.slice(0, max)}…` : text;}-export interface SwitchClientOptions {- apiUrl: string;- registrationToken: string;- gatewayAdminEmail: string;- gatewayAdminPassword: string;+interface RequestInit {+ token?: string;+ cookie?: string;+ headers?: Record<string, string>;+ body?: unknown;+ timeoutMs?: number;}export class SwitchClient {@@ -109,33 +115,33 @@ export class SwitchClient {private readonly registrationToken: string;private readonly adminEmail: string;private readonly adminPassword: string;+ private readonly environment: Environment;private gatewayCookie: string | null = null;- constructor(options: SwitchClientOptions) {- this.apiUrl = options.apiUrl;- this.registrationToken = options.registrationToken;- this.adminEmail = options.gatewayAdminEmail;- this.adminPassword = options.gatewayAdminPassword;+ constructor(env: SwitchEnv, environment: Environment) {+ this.apiUrl = env.apiUrl;+ this.registrationToken = env.agentRegistrationToken;+ this.adminEmail = env.gatewayAdminEmail;+ this.adminPassword = env.gatewayAdminPassword;+ this.environment = environment;}// ── plumbing ──────────────────────────────────────────────────────────────- private async request<T>(- method: string,- path: string,- init: { token?: string; cookie?: string; body?: unknown; timeoutMs?: number } = {}- ): Promise<T> {+ private async request<T>(method: string, path: string, init: RequestInit = {}): Promise<T> {return (await this.rawRequest(method, path, init)).parsed as T;}private async rawRequest(method: string,path: string,- init: { token?: string; cookie?: string; body?: unknown; timeoutMs?: number }+ init: RequestInit): Promise<{ parsed: unknown; response: Response }> {+ assertAllowed(this.environment, method, path);const headers: Record<string, string> = { Accept: 'application/json' };if (init.token) headers.Authorization = `Bearer ${init.token}`;if (init.cookie) headers.Cookie = init.cookie;+ Object.assign(headers, init.headers ?? {});if (init.body !== undefined) headers['Content-Type'] = 'application/json';const response = await fetch(`${this.apiUrl}${path}`, {@@ -204,13 +210,7 @@ export class SwitchClient {}async getRoom(roomId: string): Promise<RoomDetail> {- const row = await this.gateway<RawRoom>('GET', `/rooms/${roomId}`);- return {- ...toRoomSummary(row),- externalChannelId: row.external_channel_id ?? null,- matrixRoomId: row.matrix_room_id ?? null,- agentIds: row.agent_ids ?? [],- };+ return toRoomDetail(await this.gateway<RawRoom>('GET', `/rooms/${roomId}`));}/**@@ -227,29 +227,27 @@ export class SwitchClient {** `userNames` are platform usernames added to the channel — the harness adds* itself, since it has to be a channel member to post as the human.+ *+ * `bridgeId: null` makes an internal-only room: no platform channel, so the+ * only participants are agents and the fast lane needs no bridge at all.*/async createRoom(params: {name: string;description: string;- bridgeId: string;+ bridgeId: string | null;agentNames: string[];- userNames?: string[];+ userNames: string[];}): Promise<RoomDetail> {const row = await this.gateway<RawRoom>('POST', '/rooms', {name: params.name,description: params.description,- bridge_id: params.bridgeId,+ ...(params.bridgeId === null+ ? { internal_only: true }+ : { bridge_id: params.bridgeId, internal_only: false, channel_type: 'channel_public' }),agent_names: params.agentNames,- user_names: params.userNames ?? [],- channel_type: 'channel_public',- internal_only: false,+ user_names: params.userNames,});- return {- ...toRoomSummary(row),- externalChannelId: row.external_channel_id ?? null,- matrixRoomId: row.matrix_room_id ?? null,- agentIds: row.agent_ids ?? [],- };+ return toRoomDetail(row);}async deleteRoom(roomId: string): Promise<void> {@@ -265,6 +263,19 @@ export class SwitchClient {await this.gateway('DELETE', `/agents/by-name/${encodeURIComponent(name)}`);}+ /**+ * `PUT /gateway/agents/{id}/addressing-policy` with `{policy: null}` — clear+ * the agent's scoped addressing policy, so anyone may address it.+ *+ * Agents are created owner-only: only their owner, or agents and identities+ * the owner has claimed, may address them, and anything else is answered with+ * a refusal instead of delivered. The harness's senders are other throwaway+ * agents and an unclaimed platform user, so its agents are opened explicitly.+ */+ async openAddressing(agentId: string): Promise<void> {+ await this.gateway('PUT', `/agents/${agentId}/addressing-policy`, { policy: null });+ }+/*** The status Switch reports for one agent in one room. `live` means a session* is attending the room; `dormant` means an auto-session connector is watching@@ -468,6 +479,40 @@ export class SwitchClient {return body.event_id;}+ // ── agent bridge: the operations front door ───────────────────────────────++ /**+ * `GET /agents/{id}/ops` — every operation the registry defines, with its+ * parameters, keyed by name. The same registry serves the MCP server, so what this lists is+ * what an agent's session sees as tools.+ */+ async listOperations(agent: RegisteredAgent): Promise<string[]> {+ const body = await this.request<{ operations: Record<string, unknown> }>(+ 'GET',+ `/agents/${agent.id}/ops`,+ { token: agent.apiKey }+ );+ return Object.keys(body.operations);+ }++ /**+ * `POST /agents/{id}/ops/{operation}` — run one registry operation as the+ * agent, with `args` as its arguments, and return its `result`. Operations that act on "the room you+ * are connected to" need a live connection and fail without one; the fast+ * lane calls those that take the room as an argument.+ */+ async callOperation<T>(+ agent: RegisteredAgent,+ operation: string,+ args: Record<string, unknown>+ ): Promise<T> {+ const body = await this.request<{ result: T }>('POST', `/agents/${agent.id}/ops/${operation}`, {+ token: agent.apiKey,+ body: args,+ });+ return body.result;+ }+/*** `POST /agents/{id}/watch/heartbeat` — the room-agnostic "an operator* connector is watching this agent" beat. Switch Console pings it while its@@ -494,6 +539,15 @@ interface RawRoom {agent_statuses?: Record<string, unknown>;}+function toRoomDetail(row: RawRoom): RoomDetail {+ return {+ ...toRoomSummary(row),+ externalChannelId: row.external_channel_id ?? null,+ matrixRoomId: row.matrix_room_id ?? null,+ agentIds: row.agent_ids ?? [],+ };+}+function toRoomSummary(row: RawRoom): RoomSummary {return {id: row.id,diff --git a/console/tooling-e2e/src/core/core.test.ts b/console/tooling-e2e/src/core/core.test.tsnew file mode 100644index 000000000..8d2561b45--- /dev/null+++ b/console/tooling-e2e/src/core/core.test.ts@@ -0,0 +1,161 @@+import { describe, expect, it } from 'vitest';+import { FAST_SCENARIOS } from '../fast/index.ts';+import { SCENARIOS as PROVIDER_SCENARIOS } from '../provider/scenarios.ts';+import { laneFrom, loadEnv, MissingConfigError, type HarnessEnv } from './env.ts';+import { assertAllowed, ReadOnlyEnvironmentError } from './guard.ts';+import { isValidMattermostUsername, RunContext } from './run.ts';+import {+ admit,+ assertValidHeaders,+ headerProblems,+ laneOf,+ type ScenarioHeader,+} from './scenario.ts';++const BASE = {+ SWITCH_API_URL: 'http://switch.invalid/',+ SWITCH_GATEWAY_ADMIN_EMAIL: '[email protected]',+ SWITCH_GATEWAY_ADMIN_PASSWORD: 'placeholder',+ SWITCH_AGENT_REGISTRATION_TOKEN: 'placeholder',+ SWITCH_E2E_ENVIRONMENT: 'ephemeral',+};++describe('configuration', () => {+ it('needs no Mattermost, and strips a trailing slash', () => {+ const env = loadEnv('fast', BASE, {});+ expect(env.mattermost).toBeNull();+ expect(env.switch.apiUrl).toBe('http://switch.invalid');+ expect(env.strict).toBe(false);+ });++ it('names every missing variable at once', () => {+ expect(() => loadEnv('fast', {}, {})).toThrow(MissingConfigError);+ try {+ loadEnv('fast', {}, {});+ } catch (error) {+ expect((error as MissingConfigError).missing).toEqual(Object.keys(BASE));+ }+ });++ it('treats a partial Mattermost configuration as a mistake', () => {+ expect(() => loadEnv('fast', { ...BASE, MATTERMOST_URL: 'http://mm.invalid' }, {})).toThrow(+ /MATTERMOST_TOKEN, MATTERMOST_TEAM/+ );+ });++ it('refuses an unknown environment or lane', () => {+ expect(() => loadEnv('fast', { ...BASE, SWITCH_E2E_ENVIRONMENT: 'qa' }, {})).toThrow(/qa/);+ expect(() => laneFrom({ SWITCH_E2E_LANE: 'slow' })).toThrow(/slow/);+ expect(laneFrom({})).toBeNull();+ });+});++describe('read-only guard', () => {+ it('lets reads and logins through against prod, and nothing else', () => {+ expect(() => assertAllowed('prod', 'GET', '/gateway/rooms')).not.toThrow();+ expect(() => assertAllowed('prod', 'POST', '/gateway/auth/login')).not.toThrow();+ expect(() => assertAllowed('prod', 'POST', '/gateway/rooms')).toThrow(ReadOnlyEnvironmentError);+ expect(() => assertAllowed('prod', 'DELETE', '/agents/x')).toThrow(ReadOnlyEnvironmentError);+ expect(() => assertAllowed('prod', 'POST', '/gateway/auth/login/../rooms')).toThrow();+ });++ it('allows writes everywhere else', () => {+ for (const environment of ['ephemeral', 'local', 'dev', 'staging'] as const) {+ expect(() => assertAllowed(environment, 'POST', '/gateway/rooms')).not.toThrow();+ }+ });+});++describe('run naming', () => {+ it('gives each scope distinct, Mattermost-safe names', () => {+ const run = new RunContext('abc123');+ const first = run.scope();+ const second = run.scope();+ expect(first.name('snd')).toBe('e2e-abc123a-snd');+ expect(second.name('snd')).toBe('e2e-abc123b-snd');+ expect(run.owns('e2e-abc123b-snd')).toBe(true);+ expect(run.owns('e2e-zzz999a-snd')).toBe(false);+ });++ it('refuses a name too long for a bot username, or one handed out twice', () => {+ const scope = new RunContext('abc123').scope();+ expect(() => scope.name('much-too-long')).toThrow(/Shorten/);+ scope.name('room');+ expect(() => scope.name('room')).toThrow(/already/);+ });++ it('fits every provider agent role', () => {+ for (const role of ['opencode', 'antigrav', 'cursor', 'claude', 'codex']) {+ expect(isValidMattermostUsername(`e2e-abc123a-${role}`)).toBe(true);+ }+ });+});++const header = (overrides: Partial<ScenarioHeader> = {}): ScenarioHeader => ({+ id: 'some-scenario',+ covers: ['list_rooms', 'POST /agents/{agent_id}/message'],+ needsProvider: false,+ environments: ['ephemeral', 'dev'],+ owner: 'platform',+ requires: [],+ ...overrides,+});++const env = (overrides: Partial<HarnessEnv> = {}): HarnessEnv => ({+ ...loadEnv('fast', BASE, {}),+ ...overrides,+});++describe('scenario headers', () => {+ it('accepts operation names and routes, and reports every problem', () => {+ expect(headerProblems(header())).toEqual([]);+ expect(+ headerProblems(+ header({ id: 'Bad Id', covers: ['post message'], environments: [], owner: '' })+ )+ ).toHaveLength(4);+ });++ it('keeps provider scenarios off prod', () => {+ expect(headerProblems(header({ needsProvider: true, environments: ['prod'] }))).toHaveLength(1);+ });++ it('refuses a duplicated id', () => {+ expect(() => assertValidHeaders([header(), header()])).toThrow(/used by another/);+ });++ it('holds for every scenario in the suite, with ids unique across lanes', () => {+ const fast = FAST_SCENARIOS.map((scenario) => scenario.header);+ const provider = PROVIDER_SCENARIOS.map((scenario) => scenario.header);+ expect(() => assertValidHeaders([...fast, ...provider])).not.toThrow();+ expect(fast.every((h) => laneOf(h) === 'fast')).toBe(true);+ expect(provider.every((h) => laneOf(h) === 'provider')).toBe(true);+ });+});++describe('admission', () => {+ it('runs a scenario of this lane and environment with its prerequisites met', () => {+ expect(admit(header(), env())).toEqual({ run: true });+ });++ it('passes over the other lane and unlisted environments without failing', () => {+ expect(admit(header({ needsProvider: true }), env())).toMatchObject({+ run: false,+ failure: false,+ });+ expect(admit(header(), env({ environment: 'staging' }))).toMatchObject({+ run: false,+ failure: false,+ });+ });++ it('skips on a missing prerequisite, and fails on one when strict', () => {+ const needsMattermost = header({ requires: ['mattermost'] });+ expect(admit(needsMattermost, env())).toMatchObject({ run: false, failure: false });+ expect(admit(needsMattermost, env({ strict: true }))).toMatchObject({+ run: false,+ failure: true,+ reason: expect.stringContaining('MATTERMOST_URL'),+ });+ });+});diff --git a/console/tooling-e2e/src/core/env.ts b/console/tooling-e2e/src/core/env.tsnew file mode 100644index 000000000..b6346ffa1--- /dev/null+++ b/console/tooling-e2e/src/core/env.ts@@ -0,0 +1,285 @@+import { readFileSync } from 'node:fs';+import path from 'node:path';+import { fileURLToPath } from 'node:url';++/**+ * Configuration for the harness. Every value is a secret or a deployment+ * address, so nothing is hard-coded: it comes from the process environment, or+ * from a gitignored `.env` beside this package. Deployed environments' URLs are+ * deliberately absent from source.+ */++/**+ * Which runner mode a run is in.+ *+ * `fast` needs nothing but a Switch server (and Mattermost, for the scenarios+ * that cross the bridge): no provider, no Console, no LLM key. It is the lane+ * that can gate pull requests, because it carries no secrets a fork lacks.+ * `provider` needs a real agent session attending the room, driven by Switch+ * Console or its sidecar.+ */+export type Lane = 'fast' | 'provider';++/**+ * What the target is. Scenarios declare which they may run against.+ *+ * `ephemeral` is a stack created for this run and destroyed after it, so a+ * scenario may change server-global state there (feature flags, for one).+ * `local` is a developer's own long-lived stack. `prod` is read-only: the+ * clients refuse every authenticated write against it.+ */+export type Environment = 'ephemeral' | 'local' | 'dev' | 'staging' | 'prod';++/** The Switch known-agent type the provider lane registers its throwaway agent as. */+export type HarnessAgentType = 'opencode' | 'claude-code' | 'codex' | 'antigravity' | 'cursor';++/**+ * How the `question` scenario expects the agent to ask.+ *+ * `numbered` is the real thing: the provider offers a native ask-the-user tool,+ * the console relays it into the room as a numbered list, and the answer goes+ * back by number. It is the default for every agent type. `prose` is the+ * explicit fallback for a provider whose SDK sessions are offered no such tool:+ * the agent asks in an ordinary message and is answered with the word. The+ * round trip being tested (the session is still alive and still connected when+ * the second message arrives) is the same; the relay path is not exercised.+ */+export type QuestionMode = 'numbered' | 'prose';++export interface SwitchEnv {+ apiUrl: string;+ gatewayAdminEmail: string;+ gatewayAdminPassword: string;+ agentRegistrationToken: string;+}++export interface MattermostEnv {+ url: string;+ /** Token of the **human** the harness posts as. */+ token: string;+ /** Team slug (e.g. `switch`), not its display name. */+ team: string;+}++/** Settings only the provider lane reads. */+export interface ProviderEnv {+ /** Working directory the console session under test runs from. */+ agentRepoDir: string | null;+ /**+ * Where a setup records the agent and room it created, so a later process can+ * reuse them instead of registering its own. Needed because a Switch Console+ * session can only be pointed at an agent that already exists: seed first,+ * start the console, then run the scenarios against the same agent.+ */+ manifestPath: string | null;+ /** Which agent type to register, and so which provider's runtime is exercised. */+ agentType: HarnessAgentType;+ /** How `question` expects the clarifying question to arrive. */+ questionMode: QuestionMode;+}++export interface HarnessEnv {+ lane: Lane;+ environment: Environment;+ /**+ * In strict mode a scenario whose prerequisites are missing fails instead of+ * skipping. CI runs strict, so a misconfigured job goes red rather than+ * reporting a suite that ran nothing as green.+ */+ strict: boolean;+ /** Leave created agents, rooms and channels behind after the run (for debugging). */+ keepArtifacts: boolean;+ switch: SwitchEnv;+ /** `null` when no Mattermost is configured; scenarios that need one then skip or fail. */+ mattermost: MattermostEnv | null;+ provider: ProviderEnv;+}++const PACKAGE_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..', '..');++/**+ * Parse `KEY=VALUE` lines. Blank lines and `#` comments are skipped; a value may+ * be wrapped in single or double quotes. Deliberately tiny — this is the only+ * reason the package would otherwise need a dotenv dependency.+ */+export function parseDotEnv(raw: string): Record<string, string> {+ const out: Record<string, string> = {};+ for (const line of raw.split('\n')) {+ const trimmed = line.trim();+ if (trimmed === '' || trimmed.startsWith('#')) continue;+ const eq = trimmed.indexOf('=');+ if (eq <= 0) continue;+ const key = trimmed.slice(0, eq).trim();+ let value = trimmed.slice(eq + 1).trim();+ if (+ (value.startsWith('"') && value.endsWith('"')) ||+ (value.startsWith("'") && value.endsWith("'"))+ ) {+ value = value.slice(1, -1);+ }+ out[key] = value;+ }+ return out;+}++/** Values from `.env` beside this package, or `{}` when there is no such file. */+export function readDotEnvFile(file = path.join(PACKAGE_ROOT, '.env')): Record<string, string> {+ let raw: string;+ try {+ raw = readFileSync(file, 'utf8');+ } catch (error) {+ if (error instanceof Error && 'code' in error && error.code === 'ENOENT') return {};+ throw error;+ }+ return parseDotEnv(raw);+}++export class MissingConfigError extends Error {+ readonly missing: string[];+ constructor(missing: string[]) {+ super(+ `Missing required configuration: ${missing.join(', ')}. ` ++ `Set them in the environment or in console/tooling-e2e/.env (see README.md).`+ );+ this.name = 'MissingConfigError';+ this.missing = missing;+ }+}++type Source = Record<string, string | undefined>;++function reader(source: Source, file: Record<string, string>): (key: string) => string | undefined {+ return (key) => {+ const value = source[key] ?? file[key];+ return value !== undefined && value.trim() !== '' ? value.trim() : undefined;+ };+}++/**+ * The lane this process was asked to run, or `null` when none was — in which+ * case every end-to-end suite skips, which is what keeps `pnpm -r test` on a+ * developer machine from reaching for a server.+ */+export function laneFrom(source: Source = process.env): Lane | null {+ const raw = reader(source, {})('SWITCH_E2E_LANE');+ if (raw === undefined) return null;+ if (raw === 'fast' || raw === 'provider') return raw;+ throw new Error(`SWITCH_E2E_LANE must be 'fast' or 'provider' — got '${raw}'.`);+}++/**+ * Resolve the harness configuration from `process.env` layered over `.env`.+ * Throws {@link MissingConfigError} naming every absent variable at once rather+ * than failing on the first — a first run is usually missing several.+ *+ * Mattermost is all-or-nothing: none of its three variables means "no+ * Mattermost here", some of them is a mistake and is reported as one.+ */+export function loadEnv(+ lane: Lane,+ source: Source = process.env,+ file: Record<string, string> = readDotEnvFile()+): HarnessEnv {+ const get = reader(source, file);++ const required = {+ SWITCH_API_URL: get('SWITCH_API_URL'),+ SWITCH_GATEWAY_ADMIN_EMAIL: get('SWITCH_GATEWAY_ADMIN_EMAIL'),+ SWITCH_GATEWAY_ADMIN_PASSWORD: get('SWITCH_GATEWAY_ADMIN_PASSWORD'),+ SWITCH_AGENT_REGISTRATION_TOKEN: get('SWITCH_AGENT_REGISTRATION_TOKEN'),+ SWITCH_E2E_ENVIRONMENT: get('SWITCH_E2E_ENVIRONMENT'),+ };+ const mattermost = {+ MATTERMOST_URL: get('MATTERMOST_URL'),+ MATTERMOST_TOKEN: get('MATTERMOST_TOKEN'),+ MATTERMOST_TEAM: get('MATTERMOST_TEAM'),+ };+ const mattermostGiven = Object.values(mattermost).some((value) => value !== undefined);++ const missing = [+ ...Object.entries(required),+ ...(mattermostGiven ? Object.entries(mattermost) : []),+ ]+ .filter(([, value]) => value === undefined)+ .map(([key]) => key);+ if (missing.length > 0) throw new MissingConfigError(missing);++ return {+ lane,+ environment: environmentFrom(required.SWITCH_E2E_ENVIRONMENT as string),+ strict: get('SWITCH_E2E_STRICT') === '1',+ keepArtifacts: get('SWITCH_E2E_KEEP') === '1',+ switch: {+ apiUrl: stripTrailingSlash(required.SWITCH_API_URL as string),+ gatewayAdminEmail: required.SWITCH_GATEWAY_ADMIN_EMAIL as string,+ gatewayAdminPassword: required.SWITCH_GATEWAY_ADMIN_PASSWORD as string,+ agentRegistrationToken: required.SWITCH_AGENT_REGISTRATION_TOKEN as string,+ },+ mattermost: mattermostGiven+ ? {+ url: stripTrailingSlash(mattermost.MATTERMOST_URL as string),+ token: mattermost.MATTERMOST_TOKEN as string,+ team: mattermost.MATTERMOST_TEAM as string,+ }+ : null,+ provider: {+ agentRepoDir: get('SWITCH_E2E_AGENT_DIR') ?? null,+ manifestPath: get('SWITCH_E2E_MANIFEST') ?? null,+ agentType: agentTypeFrom(get('SWITCH_E2E_AGENT_TYPE')),+ questionMode: questionModeFrom(get('SWITCH_E2E_QUESTION_MODE')),+ },+ };+}++const ENVIRONMENTS: readonly Environment[] = ['ephemeral', 'local', 'dev', 'staging', 'prod'];++function environmentFrom(raw: string): Environment {+ const found = ENVIRONMENTS.find((environment) => environment === raw);+ if (!found) {+ throw new Error(+ `SWITCH_E2E_ENVIRONMENT must be one of ${ENVIRONMENTS.join(', ')} — got '${raw}'.`+ );+ }+ return found;+}++const AGENT_TYPES: readonly HarnessAgentType[] = [+ 'opencode',+ 'claude-code',+ 'codex',+ 'antigravity',+ 'cursor',+];++function agentTypeFrom(raw: string | undefined): HarnessAgentType {+ if (raw === undefined) return 'opencode';+ const found = AGENT_TYPES.find((type) => type === raw);+ if (!found) {+ throw new Error(+ `SWITCH_E2E_AGENT_TYPE must be one of ${AGENT_TYPES.join(', ')} — got '${raw}'.`+ );+ }+ return found;+}++function questionModeFrom(raw: string | undefined): QuestionMode {+ if (raw === undefined || raw === 'numbered') return 'numbered';+ if (raw === 'prose') return 'prose';+ throw new Error(`SWITCH_E2E_QUESTION_MODE must be 'numbered' or 'prose' — got '${raw}'.`);+}++export function stripTrailingSlash(url: string): string {+ return url.endsWith('/') ? url.slice(0, -1) : url;+}++/**+ * A message that names the running config without disclosing any of it. Used in+ * skip reasons and the result table, which are printed to a terminal and may be+ * pasted into a ticket.+ */+export function describeEnv(env: HarnessEnv): string {+ const mattermost = env.mattermost+ ? `mattermost=${env.mattermost.url} team=${env.mattermost.team}`+ : 'mattermost=none';+ return `lane=${env.lane} environment=${env.environment} switch=${env.switch.apiUrl} ${mattermost}`;+}diff --git a/console/tooling-e2e/src/core/guard.ts b/console/tooling-e2e/src/core/guard.tsnew file mode 100644index 000000000..9e00d0ed0--- /dev/null+++ b/console/tooling-e2e/src/core/guard.ts@@ -0,0 +1,39 @@+import type { Environment } from './env.ts';++/**+ * The read-only guard. Against `prod` the clients refuse every request that+ * could change state, before it leaves the process, so a misconfigured base URL+ * is a failed test rather than an incident.+ *+ * Logging in is the one exemption: it is a `POST` but changes nothing the+ * product shows, and a read-only smoke still has to authenticate.+ */++export class ReadOnlyEnvironmentError extends Error {+ constructor(method: string, path: string, environment: Environment) {+ super(+ `Refusing ${method} ${path}: the target environment is '${environment}', which is ` ++ `read-only for the harness. Only GET requests and logging in are allowed there.`+ );+ this.name = 'ReadOnlyEnvironmentError';+ }+}++const READ_METHODS = new Set(['GET', 'HEAD', 'OPTIONS']);++/** Paths that are writes by method but not by effect. */+const LOGIN_PATHS = new Set(['/gateway/auth/login', '/api/v4/users/login']);++export function isReadOnly(environment: Environment): boolean {+ return environment === 'prod';+}++/** Throw {@link ReadOnlyEnvironmentError} when `method path` is a write against a read-only environment. */+export function assertAllowed(environment: Environment, method: string, path: string): void {+ if (!isReadOnly(environment)) return;+ const upper = method.toUpperCase();+ if (READ_METHODS.has(upper)) return;+ const bare = path.split('?', 1)[0] ?? path;+ if (upper === 'POST' && LOGIN_PATHS.has(bare)) return;+ throw new ReadOnlyEnvironmentError(upper, bare, environment);+}diff --git a/console/tooling-e2e/src/core/run.ts b/console/tooling-e2e/src/core/run.tsnew file mode 100644index 000000000..cc796de85--- /dev/null+++ b/console/tooling-e2e/src/core/run.ts@@ -0,0 +1,95 @@+/**+ * The run id, and every name built from it.+ *+ * On a deployed environment many runs share one tenant over time, so whatever a+ * run creates has to be recognisable as that run's: list assertions filter on+ * it, teardown deletes by it, and the scheduled wipe finds leftovers by the+ * shared prefix. That only holds if no scenario ever builds a raw name itself,+ * so every name goes through {@link NameScope.name}.+ *+ * Names are `e2e-<runId><scope>-<role>`: the run, then a letter per scenario so+ * two scenarios can both ask for a `sender` without colliding, then what the+ * thing is for.+ */++/** Everything the harness creates starts with this. The cleanup script deletes by it. */+export const ARTIFACT_PREFIX = 'e2e-';++/**+ * Mattermost usernames: lowercase, 3-22 chars, `[a-z0-9._-]`, must start+ * alphanumeric. Agent names become Mattermost bot usernames verbatim, so every+ * agent name has to pass this, bridged or not.+ */+export function isValidMattermostUsername(name: string): boolean {+ return /^[a-z0-9][a-z0-9._-]{2,21}$/.test(name);+}++/** Lowercase, filesystem- and Mattermost-username-safe run id. */+export function shortId(): string {+ return Math.random().toString(36).slice(2, 8).padEnd(6, '0');+}++const SCOPE_LETTERS = 'abcdefghijklmnopqrstuvwxyz';++export class RunContext {+ readonly runId: string;+ private scopes = 0;++ constructor(runId: string = shortId()) {+ if (!/^[a-z0-9]{6}$/.test(runId)) {+ throw new Error(`Run id '${runId}' must be six lowercase alphanumerics.`);+ }+ this.runId = runId;+ }++ /** A fresh set of names for one scenario (or one setup), distinct from every other scope's. */+ scope(): NameScope {+ const letter = SCOPE_LETTERS[this.scopes];+ if (letter === undefined) {+ throw new Error(`Run ${this.runId} ran out of name scopes (${SCOPE_LETTERS.length}).`);+ }+ this.scopes += 1;+ return new NameScope(this.runId, letter);+ }++ /** Whether a name belongs to this run. */+ owns(name: string): boolean {+ return name.startsWith(`${ARTIFACT_PREFIX}${this.runId}`);+ }+}++export class NameScope {+ private readonly used = new Set<string>();++ constructor(+ readonly runId: string,+ private readonly letter: string+ ) {}++ /**+ * The name for an agent, room or channel.+ *+ * `role` is kept short because the result has to fit a Mattermost username;+ * a name that does not is refused here rather than failing bot creation on+ * the bridge, where it surfaces only as a room that is never provisioned.+ * Asking for the same role twice is refused too: two artefacts sharing a name+ * would make teardown and every by-name lookup ambiguous.+ */+ name(role: string): string {+ const name = `${ARTIFACT_PREFIX}${this.runId}${this.letter}-${role}`;+ if (!isValidMattermostUsername(name)) {+ throw new Error(+ `Derived name '${name}' is not a valid Mattermost username ` ++ `(lowercase, 3-22 chars, [a-z0-9._-]). Shorten the role '${role}'.`+ );+ }+ if (this.used.has(name)) throw new Error(`Name '${name}' was already handed out.`);+ this.used.add(name);+ return name;+ }++ /** A marker string unique to this scope, for grepping a transcript. */+ marker(label: string): string {+ return `SWITCH_E2E_${label.toUpperCase()}_${this.runId.toUpperCase()}${this.letter.toUpperCase()}`;+ }+}diff --git a/console/tooling-e2e/src/core/scenario.ts b/console/tooling-e2e/src/core/scenario.tsnew file mode 100644index 000000000..b7f5d1944--- /dev/null+++ b/console/tooling-e2e/src/core/scenario.ts@@ -0,0 +1,128 @@+import type { Environment, HarnessEnv, Lane } from './env.ts';++/**+ * The machine-readable header every scenario carries, in either lane.+ *+ * It is what lets tooling answer questions about the suite without running it:+ * which lane a scenario belongs to (`needsProvider`), where it may run+ * (`environments`), who to ask when it breaks (`owner`), and what it exercises+ * (`covers`), which is what the coverage gate reads.+ */+export interface ScenarioHeader {+ /** Kebab-case, unique across the suite. */+ id: string;+ /**+ * What the scenario exercises: agent-operation names from the registry+ * (`core/switch_core/bridges/agent/operations/definitions.py`, e.g.+ * `list_rooms`) and HTTP routes as `METHOD /path/{param}` with the server's+ * own parameter names (e.g. `POST /agents/{agent_id}/message`).+ */+ covers: readonly string[];+ /** `true` puts the scenario in the provider lane: it needs a real agent session. */+ needsProvider: boolean;+ /**+ * Where it may run. A scenario that changes server-global state (a feature+ * flag) lists only `ephemeral`. One that writes anything never lists `prod`.+ */+ environments: readonly Environment[];+ /** Who answers for it — a team or role, not a person. */+ owner: string;+ /** What has to be configured for it to mean anything. */+ requires: readonly Prerequisite[];+}++/**+ * `mattermost`: a Mattermost and a human token on it are configured.+ * `agent-dir`: the provider session's working directory is on this machine.+ */+export type Prerequisite = 'mattermost' | 'agent-dir';++const OPERATION = /^[a-z][a-z0-9_]*$/;+const ROUTE = /^(GET|POST|PUT|PATCH|DELETE) \/\S*$/;+const ID = /^[a-z0-9]+(-[a-z0-9]+)*$/;++/** Every problem with a header, so a bad one is fixed in one pass. Empty when it is valid. */+export function headerProblems(header: ScenarioHeader): string[] {+ const problems: string[] = [];+ if (!ID.test(header.id)) problems.push(`id '${header.id}' is not kebab-case`);+ if (header.covers.length === 0) problems.push('covers is empty');+ for (const entry of header.covers) {+ if (!OPERATION.test(entry) && !ROUTE.test(entry)) {+ problems.push(`covers entry '${entry}' is neither an operation name nor 'METHOD /path'`);+ }+ }+ if (header.environments.length === 0) problems.push('environments is empty');+ if (header.owner.trim() === '') problems.push('owner is empty');+ if (header.needsProvider && header.environments.includes('prod')) {+ problems.push('a provider scenario writes to the room, so it cannot list prod');+ }+ return problems;+}++/** Throw naming every problem in every header, plus any id used twice. */+export function assertValidHeaders(headers: readonly ScenarioHeader[]): void {+ const problems: string[] = [];+ const seen = new Set<string>();+ for (const header of headers) {+ for (const problem of headerProblems(header)) problems.push(`${header.id}: ${problem}`);+ if (seen.has(header.id)) problems.push(`${header.id}: id used by another scenario`);+ seen.add(header.id);+ }+ if (problems.length > 0) {+ throw new Error(`Invalid scenario headers:\n ${problems.join('\n ')}`);+ }+}++export function laneOf(header: ScenarioHeader): Lane {+ return header.needsProvider ? 'provider' : 'fast';+}++/**+ * Whether a scenario runs here, and if not, whether that is a failure.+ *+ * A scenario from the other lane, or one that does not list this environment,+ * is simply not this run's business. A missing prerequisite is different: the+ * scenario belongs here and cannot run, which skips on a developer machine and+ * fails in strict mode — CI must not report a scenario that never ran as green.+ */+export type Admission = { run: true } | { run: false; reason: string; failure: boolean };++export function admit(header: ScenarioHeader, env: HarnessEnv): Admission {+ if (laneOf(header) !== env.lane) {+ return { run: false, reason: `belongs to the ${laneOf(header)} lane`, failure: false };+ }+ if (!header.environments.includes(env.environment)) {+ return {+ run: false,+ reason: `does not run against '${env.environment}' (only ${header.environments.join(', ')})`,+ failure: false,+ };+ }+ const missing = header.requires.filter((prerequisite) => !satisfied(prerequisite, env));+ if (missing.length > 0) {+ return {+ run: false,+ reason: `needs ${missing.map(describePrerequisite).join(' and ')}`,+ failure: env.strict,+ };+ }+ return { run: true };+}++function satisfied(prerequisite: Prerequisite, env: HarnessEnv): boolean {+ switch (prerequisite) {+ case 'mattermost':+ return env.mattermost !== null;+ case 'agent-dir':+ return env.provider.agentRepoDir !== null;+ }+}++function describePrerequisite(prerequisite: Prerequisite): string {+ switch (prerequisite) {+ case 'mattermost':+ return 'Mattermost (MATTERMOST_URL, MATTERMOST_TOKEN, MATTERMOST_TEAM)';+ case 'agent-dir':+ return 'the session working directory (SWITCH_E2E_AGENT_DIR)';+ }+}diff --git a/console/tooling-e2e/src/core/suite.ts b/console/tooling-e2e/src/core/suite.tsnew file mode 100644index 000000000..18c6ddf99--- /dev/null+++ b/console/tooling-e2e/src/core/suite.ts@@ -0,0 +1,70 @@+import { MattermostClient } from '../clients/mattermost-client.ts';+import { SwitchClient } from '../clients/switch-client.ts';+import { laneFrom, loadEnv, type HarnessEnv, type Lane } from './env.ts';++/**+ * Whether this process runs a lane's suite at all, and the clients it runs with.+ */+export interface LaneTarget {+ env: HarnessEnv;+ switchClient: SwitchClient;+ mattermost: MattermostClient | null;+}++export type LaneResolution = { target: LaneTarget; skip: null } | { target: null; skip: string };++/**+ * Resolve a lane: configuration, then reachability.+ *+ * Not asked for this lane (`SWITCH_E2E_LANE` unset or naming the other one) is+ * always a skip — that is `pnpm -r test` on a developer machine. Asked for it+ * but unable to run (configuration missing, a server unreachable) skips with the+ * reason printed, or **throws** under `SWITCH_E2E_STRICT=1`, so CI cannot pass a+ * suite that never ran.+ */+export async function resolveLane(lane: Lane): Promise<LaneResolution> {+ const asked = laneFrom();+ if (asked !== lane) {+ return {+ target: null,+ skip: asked === null ? 'SWITCH_E2E_LANE is not set' : `SWITCH_E2E_LANE is '${asked}'`,+ };+ }+ const strict = process.env.SWITCH_E2E_STRICT === '1';+ const unable = (reason: string): LaneResolution => {+ if (strict) throw new Error(`The ${lane} lane cannot run (SWITCH_E2E_STRICT=1): ${reason}`);+ return { target: null, skip: reason };+ };++ let env: HarnessEnv;+ try {+ env = loadEnv(lane);+ } catch (error) {+ return unable(message(error));+ }++ const switchClient = new SwitchClient(env.switch, env.environment);+ try {+ await switchClient.health();+ } catch (error) {+ return unable(`Switch server at ${env.switch.apiUrl} is unreachable: ${message(error)}`);+ }++ let mattermost: MattermostClient | null = null;+ if (env.mattermost) {+ mattermost = new MattermostClient(env.mattermost, env.environment);+ try {+ await mattermost.me();+ } catch (error) {+ return unable(+ `Mattermost at ${env.mattermost.url} is unreachable or MATTERMOST_TOKEN is invalid: ${message(error)}`+ );+ }+ }++ return { target: { env, switchClient, mattermost }, skip: null };+}++export function message(error: unknown): string {+ return error instanceof Error ? error.message : String(error);+}diff --git a/console/tooling-e2e/src/env.ts b/console/tooling-e2e/src/env.tsdeleted file mode 100644index f24dad4de..000000000--- a/console/tooling-e2e/src/env.ts+++ /dev/null@@ -1,177 +0,0 @@-import { readFileSync } from 'node:fs';-import path from 'node:path';-import { fileURLToPath } from 'node:url';--/**- * Configuration for the harness. Every value is a secret or a deployment- * address, so nothing is hard-coded: it comes from the process environment, or- * from a gitignored `.env` beside this package.- */-/** The Switch known-agent type the harness registers its throwaway agent as. */-export type HarnessAgentType = 'opencode' | 'claude-code' | 'codex' | 'antigravity' | 'cursor';--/**- * How the `question` scenario expects the agent to ask.- *- * `numbered` is the real thing: the provider offers a native ask-the-user tool,- * the console relays it into the room as a numbered list, and the answer goes- * back by number. It is the default for every agent type. `prose` is the- * explicit fallback for a provider whose SDK sessions are offered no such tool:- * the agent asks in an ordinary message and is answered with the word. The- * round trip being tested (the session is still alive and still connected when- * the second message arrives) is the same; the relay path is not exercised.- */-export type QuestionMode = 'numbered' | 'prose';--export interface HarnessEnv {- switchApiUrl: string;- gatewayAdminEmail: string;- gatewayAdminPassword: string;- agentRegistrationToken: string;- mattermostUrl: string;- mattermostToken: string;- mattermostTeam: string;- /** Working directory the console session under test runs OpenCode from. */- agentRepoDir: string | null;- /**- * Where a setup records the agent and room it created, so a later process can- * reuse them instead of registering its own. Needed because a Switch Console- * session can only be pointed at an agent that already exists: seed first,- * start the console, then run the scenarios against the same agent.- */- manifestPath: string | null;- /** Leave the channel and agent behind after the run (for debugging). */- keepArtifacts: boolean;- /** Which agent type to register, and so which provider's runtime is exercised. */- agentType: HarnessAgentType;- /** How `question` expects the clarifying question to arrive. */- questionMode: QuestionMode;-}--const PACKAGE_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');--/**- * Parse `KEY=VALUE` lines. Blank lines and `#` comments are skipped; a value may- * be wrapped in single or double quotes. Deliberately tiny — this is the only- * reason the package would otherwise need a dotenv dependency.- */-export function parseDotEnv(raw: string): Record<string, string> {- const out: Record<string, string> = {};- for (const line of raw.split('\n')) {- const trimmed = line.trim();- if (trimmed === '' || trimmed.startsWith('#')) continue;- const eq = trimmed.indexOf('=');- if (eq <= 0) continue;- const key = trimmed.slice(0, eq).trim();- let value = trimmed.slice(eq + 1).trim();- if (- (value.startsWith('"') && value.endsWith('"')) ||- (value.startsWith("'") && value.endsWith("'"))- ) {- value = value.slice(1, -1);- }- out[key] = value;- }- return out;-}--/** Values from `.env` beside this package, or `{}` when there is no such file. */-export function readDotEnvFile(file = path.join(PACKAGE_ROOT, '.env')): Record<string, string> {- try {- return parseDotEnv(readFileSync(file, 'utf8'));- } catch {- return {};- }-}--export class MissingConfigError extends Error {- readonly missing: string[];- constructor(missing: string[]) {- super(- `Missing required configuration: ${missing.join(', ')}. ` +- `Set them in the environment or in console/tooling-e2e/.env (see README.md).`- );- this.name = 'MissingConfigError';- this.missing = missing;- }-}--/**- * Resolve the harness configuration from `process.env` layered over `.env`.- * Throws {@link MissingConfigError} naming every absent variable at once rather- * than failing on the first — a first run is usually missing several.- */-export function loadEnv(source: Record<string, string | undefined> = process.env): HarnessEnv {- const file = readDotEnvFile();- const get = (key: string): string | undefined => {- const value = source[key] ?? file[key];- return value !== undefined && value.trim() !== '' ? value.trim() : undefined;- };-- const required = {- SWITCH_API_URL: get('SWITCH_API_URL'),- SWITCH_GATEWAY_ADMIN_EMAIL: get('SWITCH_GATEWAY_ADMIN_EMAIL'),- SWITCH_GATEWAY_ADMIN_PASSWORD: get('SWITCH_GATEWAY_ADMIN_PASSWORD'),- SWITCH_AGENT_REGISTRATION_TOKEN: get('SWITCH_AGENT_REGISTRATION_TOKEN'),- MATTERMOST_URL: get('MATTERMOST_URL'),- MATTERMOST_TOKEN: get('MATTERMOST_TOKEN'),- MATTERMOST_TEAM: get('MATTERMOST_TEAM'),- };-- const missing = Object.entries(required)- .filter(([, value]) => value === undefined)- .map(([key]) => key);- if (missing.length > 0) throw new MissingConfigError(missing);-- return {- switchApiUrl: stripTrailingSlash(required.SWITCH_API_URL as string),- gatewayAdminEmail: required.SWITCH_GATEWAY_ADMIN_EMAIL as string,- gatewayAdminPassword: required.SWITCH_GATEWAY_ADMIN_PASSWORD as string,- agentRegistrationToken: required.SWITCH_AGENT_REGISTRATION_TOKEN as string,- mattermostUrl: stripTrailingSlash(required.MATTERMOST_URL as string),- mattermostToken: required.MATTERMOST_TOKEN as string,- mattermostTeam: required.MATTERMOST_TEAM as string,- agentRepoDir: get('SWITCH_E2E_AGENT_DIR') ?? null,- manifestPath: get('SWITCH_E2E_MANIFEST') ?? null,- keepArtifacts: (get('SWITCH_E2E_KEEP') ?? '') === '1',- agentType: agentTypeFrom(get('SWITCH_E2E_AGENT_TYPE')),- questionMode: get('SWITCH_E2E_QUESTION_MODE') === 'prose' ? 'prose' : 'numbered',- };-}--const AGENT_TYPES: readonly HarnessAgentType[] = [- 'opencode',- 'claude-code',- 'codex',- 'antigravity',- 'cursor',-];--function agentTypeFrom(raw: string | undefined): HarnessAgentType {- if (raw === undefined) return 'opencode';- const found = AGENT_TYPES.find((type) => type === raw);- if (!found) {- throw new Error(- `SWITCH_E2E_AGENT_TYPE must be one of ${AGENT_TYPES.join(', ')} — got '${raw}'.`- );- }- return found;-}--export function stripTrailingSlash(url: string): string {- return url.endsWith('/') ? url.slice(0, -1) : url;-}--/** Lowercase, filesystem- and Mattermost-username-safe run id. */-export function shortId(): string {- return Math.random().toString(36).slice(2, 8);-}--/**- * A message that names the running config without disclosing any of it. Used in- * skip reasons and the result table, which are printed to a terminal and may be- * pasted into a ticket.- */-export function describeEnv(env: HarnessEnv): string {- return `switch=${env.switchApiUrl} mattermost=${env.mattermostUrl} team=${env.mattermostTeam}`;-}diff --git a/console/tooling-e2e/src/fast/agent-messaging.ts b/console/tooling-e2e/src/fast/agent-messaging.tsnew file mode 100644index 000000000..ccd2203b2--- /dev/null+++ b/console/tooling-e2e/src/fast/agent-messaging.ts@@ -0,0 +1,103 @@+import { expect } from 'vitest';+import { eventText, sleep } from '../clients/switch-client.ts';+import { WRITABLE, type FastScenario } from './scenario.ts';++/**+ * Agent-to-agent messaging in a room with no bridge: the core of the bus with+ * nothing in front of it. Two throwaway agents, one internal-only room, the+ * harness acting as both.+ */++const NOTIFICATION_DEADLINE_MS = 30_000;++/**+ * An addressed message reaches its recipient's notification stream, carrying+ * the room, the sender, and the addressed flag.+ */+export const addressedDelivery: FastScenario = {+ header: {+ id: 'addressed-delivery',+ covers: [+ 'POST /agents/register-known',+ 'POST /gateway/rooms',+ 'POST /agents/{agent_id}/message',+ 'GET /agents/{agent_id}/notifications',+ ],+ needsProvider: false,+ environments: WRITABLE,+ owner: 'platform',+ requires: [],+ },+ async run({ fixtures, names, switchClient }) {+ const sender = await fixtures.agent('snd');+ const recipient = await fixtures.agent('rcv');+ const room = await fixtures.internalRoom('room', [sender, recipient]);+ expect(room.agentIds).toEqual(expect.arrayContaining([sender.id, recipient.id]));++ const marker = names.marker('addr');+ const eventId = await switchClient.sendMessage(sender, room.id, `@${recipient.name} ${marker}`);+ expect(eventId).toBeTruthy();++ const { match, seen } = await switchClient.waitForNotification(+ recipient,+ (event) => eventText(event).includes(marker),+ NOTIFICATION_DEADLINE_MS+ );+ expect(+ match,+ `no notification carried ${marker}; saw ${JSON.stringify(seen).slice(0, 800)}`+ ).not.toBeNull();+ expect(match!.type).toBe('message');+ expect(match!.room_id).toBe(room.id);+ expect(match!.payload?.addressed).toBe(true);+ expect(match!.payload?.sender_name).toBe(sender.name);+ },+};++/**+ * Unaddressed messages are history, not notifications: the recipient's room+ * history holds every one of them, in the order they were sent, and its+ * notification stream holds none.+ *+ * Ordering is asserted because the bus promises it — `seq` is assigned under a+ * per-room lock — and a regression there is invisible to every test that sends+ * one message.+ */+export const unaddressedHistory: FastScenario = {+ header: {+ id: 'unaddressed-history',+ covers: [+ 'POST /agents/{agent_id}/message',+ 'GET /agents/{agent_id}/rooms/{room_id}/history',+ 'GET /agents/{agent_id}/notifications',+ ],+ needsProvider: false,+ environments: WRITABLE,+ owner: 'platform',+ requires: [],+ },+ async run({ fixtures, names, switchClient }) {+ const sender = await fixtures.agent('snd');+ const reader = await fixtures.agent('rdr');+ const room = await fixtures.internalRoom('room', [sender, reader]);++ const marker = names.marker('chat');+ const sent = Array.from({ length: 5 }, (_, index) => `${marker} #${index + 1}`);+ for (const body of sent) await switchClient.sendMessage(sender, room.id, body);++ let bodies: string[] = [];+ const until = Date.now() + NOTIFICATION_DEADLINE_MS;+ while (Date.now() < until) {+ const history = await switchClient.roomHistory(reader, room.id, 50);+ bodies = history.map((event) => event.body).filter((body) => body.includes(marker));+ if (bodies.length >= sent.length) break;+ await sleep(1_000);+ }+ expect(bodies).toEqual(sent);++ const notified = (await switchClient.pollNotifications(reader, 3)).filter((event) =>+ eventText(event).includes(marker)+ );+ expect(notified, 'an unaddressed message was delivered as a notification').toEqual([]);+ },+};diff --git a/console/tooling-e2e/src/fast/fast.e2e.test.ts b/console/tooling-e2e/src/fast/fast.e2e.test.tsnew file mode 100644index 000000000..1440f8827--- /dev/null+++ b/console/tooling-e2e/src/fast/fast.e2e.test.ts@@ -0,0 +1,56 @@+import { beforeAll, describe, expect, it } from 'vitest';+import { describeEnv } from '../core/env.ts';+import { RunContext } from '../core/run.ts';+import { admit, assertValidHeaders } from '../core/scenario.ts';+import { resolveLane, type LaneResolution } from '../core/suite.ts';+import { Fixtures } from '../fixtures/fixtures.ts';+import { FAST_SCENARIOS } from './index.ts';++/**+ * The fast lane: provider-free scenarios against a Switch server, one fresh+ * set of agents and rooms per scenario. Runs when `SWITCH_E2E_LANE=fast`.+ *+ * A scenario that cannot run here is reported as **skipped**, with the reason,+ * never as passed — and under `SWITCH_E2E_STRICT=1` a missing prerequisite is a+ * failure instead.+ */++assertValidHeaders(FAST_SCENARIOS.map((scenario) => scenario.header));++const run = new RunContext();+let lane: LaneResolution;++beforeAll(async () => {+ lane = await resolveLane('fast');+ if (lane.target) console.log(`fast lane run ${run.runId}: ${describeEnv(lane.target.env)}`);+}, 60_000);++describe('fast lane', () => {+ for (const scenario of FAST_SCENARIOS) {+ it(scenario.header.id, async (context) => {+ if (!lane.target) return context.skip(lane.skip);+ const { env, switchClient, mattermost } = lane.target;++ const admission = admit(scenario.header, env);+ if (!admission.run) {+ if (admission.failure) {+ throw new Error(+ `${scenario.header.id} cannot run (SWITCH_E2E_STRICT=1): ${admission.reason}`+ );+ }+ return context.skip(admission.reason);+ }++ const names = run.scope();+ const fixtures = new Fixtures(env, names, switchClient, mattermost);+ const log = (message: string): void => console.log(`[${scenario.header.id}] ${message}`);+ try {+ await scenario.run({ env, names, switchClient, mattermost, fixtures, log });+ } finally {+ const leftovers = await fixtures.teardown();+ for (const line of leftovers) log(`teardown: ${line}`);+ if (!env.keepArtifacts) expect(leftovers, 'teardown left artefacts behind').toEqual([]);+ }+ });+ }+});diff --git a/console/tooling-e2e/src/fast/index.ts b/console/tooling-e2e/src/fast/index.tsnew file mode 100644index 000000000..c7b7347de--- /dev/null+++ b/console/tooling-e2e/src/fast/index.ts@@ -0,0 +1,12 @@+import { addressedDelivery, unaddressedHistory } from './agent-messaging.ts';+import { mattermostRoundtrip } from './mattermost-roundtrip.ts';+import { operationsFrontDoor } from './operations.ts';+import type { FastScenario } from './scenario.ts';++/** Every fast-lane scenario. A scenario not listed here does not run. */+export const FAST_SCENARIOS: readonly FastScenario[] = [+ addressedDelivery,+ unaddressedHistory,+ operationsFrontDoor,+ mattermostRoundtrip,+];diff --git a/console/tooling-e2e/src/fast/mattermost-roundtrip.ts b/console/tooling-e2e/src/fast/mattermost-roundtrip.tsnew file mode 100644index 000000000..4c09d31b3--- /dev/null+++ b/console/tooling-e2e/src/fast/mattermost-roundtrip.ts@@ -0,0 +1,77 @@+import { expect } from 'vitest';+import { eventText } from '../clients/switch-client.ts';+import { WRITABLE, type FastScenario } from './scenario.ts';++/**+ * The bridge round trip with **no Switch Console and no model in the loop**:+ * the harness plays both the human and the agent.+ *+ * 1. create a room on the Mattermost bridge and confirm Switch provisions its+ * channel with the agent in it;+ * 2. post `@agent …` as a human and confirm the addressed message reaches the+ * agent-facing notification stream;+ * 3. `POST /agents/{id}/message` as the agent and confirm it lands back in the+ * Mattermost channel as the agent's bot.+ *+ * If this passes and a provider-lane scenario fails, the fault is in the agent+ * session, not in Switch, the bridge, or the harness.+ */+export const mattermostRoundtrip: FastScenario = {+ header: {+ id: 'mattermost-roundtrip',+ covers: [+ 'POST /gateway/rooms',+ 'GET /agents/{agent_id}/notifications',+ 'POST /agents/{agent_id}/message',+ ],+ needsProvider: false,+ environments: WRITABLE,+ owner: 'platform',+ requires: ['mattermost'],+ },+ async run({ fixtures, names, switchClient, mattermost, log }) {+ const agent = await fixtures.agent('loop');+ const bridged = await fixtures.mattermostRoom('room', [agent]);+ const bot = bridged.bots.get(agent.id)!;+ expect(bridged.room.externalChannelId).toBe(bridged.channel.id);+ expect(bridged.room.agentIds).toContain(agent.id);++ const marker = names.marker('loop');+ const sinceMs = Date.now();+ const posted = await mattermost!.post({+ channelId: bridged.channel.id,+ message: `@${agent.name} hello ${marker}`,+ });+ log(`posted ${posted.id} as ${bridged.human.username}`);++ // ── inbound: room -> Switch -> agent ────────────────────────────────────+ const inbound = await switchClient.waitForNotification(+ agent,+ (event) => eventText(event).includes(marker),+ 120_000+ );+ expect(+ inbound.match,+ `the addressed message never reached the agent; saw ${JSON.stringify(inbound.seen).slice(0, 800)}`+ ).not.toBeNull();+ expect(inbound.match!.room_id).toBe(bridged.room.id);+ expect(inbound.match!.payload?.addressed).toBe(true);++ // ── outbound: agent -> Switch -> room ───────────────────────────────────+ const eventId = await switchClient.sendMessage(agent, bridged.room.id, `ack ${marker}`);+ expect(eventId).toBeTruthy();++ const outbound = await mattermost!.waitForPost({+ channelId: bridged.channel.id,+ fromUserId: bot.id,+ predicate: (post) => post.message.includes(`ack ${marker}`),+ sinceMs,+ deadlineMs: 120_000,+ });+ expect(+ outbound.match,+ `the agent's reply never appeared as bot ${bot.username}; channel said: ` ++ outbound.transcript.map((post) => `${post.user_id}: ${post.message}`).join(' | ')+ ).not.toBeNull();+ },+};diff --git a/console/tooling-e2e/src/fast/operations.ts b/console/tooling-e2e/src/fast/operations.tsnew file mode 100644index 000000000..5a63dc18f--- /dev/null+++ b/console/tooling-e2e/src/fast/operations.ts@@ -0,0 +1,48 @@+import { expect } from 'vitest';+import { WRITABLE, type FastScenario } from './scenario.ts';++/**+ * The agent-operations front door over HTTP: the same registry the MCP server+ * serves, called the way a session host's tools are.+ */+export const operationsFrontDoor: FastScenario = {+ header: {+ id: 'operations-front-door',+ covers: [+ 'GET /agents/{agent_id}/ops',+ 'POST /agents/{agent_id}/ops/{operation}',+ 'list_rooms',+ 'read_context',+ ],+ needsProvider: false,+ environments: WRITABLE,+ owner: 'platform',+ requires: [],+ },+ async run({ fixtures, names, switchClient }) {+ const agent = await fixtures.agent('ops');+ const peer = await fixtures.agent('peer');+ const room = await fixtures.internalRoom('room', [agent, peer]);++ const operations = await switchClient.listOperations(agent);+ expect(operations).toEqual(+ expect.arrayContaining(['list_rooms', 'read_context', 'post_message'])+ );++ const rooms = await switchClient.callOperation<{ room_id: string; name: string }[]>(+ agent,+ 'list_rooms',+ {}+ );+ expect(rooms).toContainEqual(expect.objectContaining({ room_id: room.id, name: room.name }));++ const marker = names.marker('ops');+ await switchClient.sendMessage(peer, room.id, marker);+ const context = await switchClient.callOperation<{+ threads: { root: { body: string; sender_name: string } }[];+ }>(agent, 'read_context', { room_id: room.id, limit: 20 });+ expect(context.threads.map((thread) => thread.root)).toContainEqual(+ expect.objectContaining({ body: marker, sender_name: peer.name })+ );+ },+};diff --git a/console/tooling-e2e/src/fast/scenario.ts b/console/tooling-e2e/src/fast/scenario.tsnew file mode 100644index 000000000..0df658c1e--- /dev/null+++ b/console/tooling-e2e/src/fast/scenario.ts@@ -0,0 +1,32 @@+import type { MattermostClient } from '../clients/mattermost-client.ts';+import type { SwitchClient } from '../clients/switch-client.ts';+import type { HarnessEnv } from '../core/env.ts';+import type { NameScope } from '../core/run.ts';+import type { ScenarioHeader } from '../core/scenario.ts';+import type { Fixtures } from '../fixtures/fixtures.ts';++/**+ * A fast-lane scenario: plain code, run against a Switch server over HTTP with+ * the harness playing every participant. No provider, no Console, no model.+ *+ * `run` asserts with `expect` and throws on failure. Everything it creates goes+ * through `fixtures`, which tears it down afterwards whether or not it passed.+ */+export interface FastScenario {+ header: ScenarioHeader;+ run(context: FastContext): Promise<void>;+}++export interface FastContext {+ env: HarnessEnv;+ /** Names and markers for this scenario, unique within the run. */+ names: NameScope;+ switchClient: SwitchClient;+ /** Present exactly when the header requires `mattermost`. */+ mattermost: MattermostClient | null;+ fixtures: Fixtures;+ log: (message: string) => void;+}++/** Where a fast scenario that writes may run: anywhere but prod. */+export const WRITABLE: ScenarioHeader['environments'] = ['ephemeral', 'local', 'dev', 'staging'];diff --git a/console/tooling-e2e/src/fixtures/fixtures.ts b/console/tooling-e2e/src/fixtures/fixtures.tsnew file mode 100644index 000000000..51da21e7b--- /dev/null+++ b/console/tooling-e2e/src/fixtures/fixtures.ts@@ -0,0 +1,237 @@+import type {+ MattermostChannel,+ MattermostClient,+ MattermostUser,+} from '../clients/mattermost-client.ts';+import {+ sleep,+ type RegisteredAgent,+ type RoomDetail,+ type SwitchClient,+} from '../clients/switch-client.ts';+import type { HarnessAgentType, HarnessEnv } from '../core/env.ts';+import type { NameScope } from '../core/run.ts';++/**+ * What a scenario stands up, and the teardown that removes it.+ *+ * Every artefact is named through the run context and registered for teardown+ * the moment it exists, so a scenario that fails halfway still cleans up what it+ * made. Teardown runs newest first — rooms before the agents in them — and+ * reports every failure rather than stopping at the first, because one leaked+ * room should not hide another.+ */+export class Fixtures {+ private readonly cleanups: { label: string; run: () => Promise<void> }[] = [];++ constructor(+ readonly env: HarnessEnv,+ readonly names: NameScope,+ readonly switchClient: SwitchClient,+ readonly mattermost: MattermostClient | null+ ) {}++ /**+ * Register an agent named for `role` in this scope.+ *+ * `autoSession` is off unless asked for: with it on, Switch expects a+ * connector to spawn a session when the agent is addressed, and an operator's+ * Console watching the same server could fight the harness for the room.+ */+ async agent(+ role: string,+ options: { autoSession?: boolean; agentType?: HarnessAgentType; repoDir?: string } = {}+ ): Promise<RegisteredAgent> {+ const name = this.names.name(role);+ const agent = await this.switchClient.registerKnownAgent({+ agentType: options.agentType ?? 'opencode',+ name,+ description: `Switch end-to-end harness agent (run ${this.names.runId}). Safe to delete.`,+ options: {+ auto_session: options.autoSession ?? false,+ ...(options.repoDir ? { repo_dir: options.repoDir } : {}),+ },+ });+ this.defer(`agent ${name} (${agent.id})`, () => this.switchClient.deleteAgent(agent));+ await this.switchClient.openAddressing(agent.id);+ return agent;+ }++ /** A room with no platform channel: agents only, no bridge involved. */+ async internalRoom(role: string, agents: RegisteredAgent[]): Promise<RoomDetail> {+ const name = this.names.name(role);+ const room = await this.switchClient.createRoom({+ name,+ description: `Switch end-to-end harness (run ${this.names.runId}). Safe to delete.`,+ bridgeId: null,+ agentNames: agents.map((agent) => agent.name),+ userNames: [],+ });+ this.defer(`room ${name} (${room.id})`, () => this.switchClient.deleteRoom(room.id));+ if (room.externalChannelId !== null) {+ throw new Error(`Internal-only room ${room.id} came back with a platform channel.`);+ }+ return room;+ }++ /**+ * A room on the Mattermost bridge, with the human the harness posts as in its+ * channel, and the bridge proven live in both directions' first hop.+ *+ * Created from the gateway so Switch provisions the channel and adds the bots+ * itself. The other route — make a channel, add a bot — creates no room when+ * the bot is the first in a new channel, because nobody's websocket witnesses+ * the join.+ */+ async mattermostRoom(role: string, agents: RegisteredAgent[]): Promise<BridgedRoom> {+ const mattermost = this.requireMattermost();+ const human = await mattermost.me();+ const bridgeId = await this.switchClient.defaultBridgeId('mattermost');+ // Resolved even though room creation does not need it: it fails fast and+ // clearly when MATTERMOST_TEAM names a team that does not exist.+ await mattermost.findTeam(this.mattermostTeam());++ // Registration mints each agent's bot; wait for them so the room's members+ // and any later `@name` mention have accounts to resolve to.+ const bots = new Map<string, MattermostUser>();+ for (const agent of agents) {+ bots.set(agent.id, await mattermost.waitForBotUser(agent.name, 60_000));+ }++ const name = this.names.name(role);+ const room = await this.switchClient.createRoom({+ name,+ description: `Switch end-to-end harness (run ${this.names.runId}). Safe to delete.`,+ bridgeId,+ agentNames: agents.map((agent) => agent.name),+ userNames: [human.username],+ });+ this.defer(`room ${name} (${room.id})`, () => this.switchClient.deleteRoom(room.id));+ if (!room.externalChannelId) {+ throw new Error(+ `Switch room ${room.id} was created without a Mattermost channel — ` ++ `the collaboration bridge did not provision one.`+ );+ }+ const channelId = room.externalChannelId;+ // Deleting the room usually takes the channel with it, so archiving is+ // normally a no-op; it is the safety net when it does not.+ this.defer(`channel ${channelId}`, async () => {+ await mattermost.archiveChannel(channelId).catch(() => undefined);+ });++ const channel = await mattermost.getChannel(channelId);+ const bridged: BridgedRoom = { room, channel, human, bots };+ const witness = agents[0];+ if (!witness) throw new Error('A bridged room needs at least one agent.');+ await waitForBridgeReady({+ switchClient: this.switchClient,+ mattermost,+ agent: witness,+ room,+ channel,+ runId: this.names.runId,+ });+ return bridged;+ }++ /** Register a cleanup step for something created outside these helpers. */+ defer(label: string, run: () => Promise<void>): void {+ this.cleanups.push({ label, run });+ }++ /**+ * Undo everything, newest first. Never throws: a teardown failure must not+ * mask the scenario's result. Returns what could not be removed, so the+ * caller can print it and an operator can clean up by hand.+ */+ async teardown(): Promise<string[]> {+ if (this.env.keepArtifacts) {+ const kept = this.cleanups.map((cleanup) => cleanup.label);+ this.cleanups.length = 0;+ return kept.map((label) => `kept (SWITCH_E2E_KEEP=1): ${label}`);+ }+ const failures: string[] = [];+ while (this.cleanups.length > 0) {+ const cleanup = this.cleanups.pop()!;+ try {+ await cleanup.run();+ } catch (error) {+ failures.push(+ `could not remove ${cleanup.label}: ${error instanceof Error ? error.message : String(error)}`+ );+ }+ }+ return failures;+ }++ private requireMattermost(): MattermostClient {+ if (!this.mattermost) {+ throw new Error(+ 'This scenario needs Mattermost but none is configured — declare ' ++ "`requires: ['mattermost']` in its header so it is skipped or failed up front."+ );+ }+ return this.mattermost;+ }++ private mattermostTeam(): string {+ if (!this.env.mattermost) throw new Error('No Mattermost configured.');+ return this.env.mattermost.team;+ }+}++export interface BridgedRoom {+ room: RoomDetail;+ channel: MattermostChannel;+ /** The human the harness posts as. */+ human: MattermostUser;+ /** Each agent's Mattermost bot, by agent id. A bot's username is the agent's name. */+ bots: Map<string, MattermostUser>;+}++/**+ * Block until a plain channel message actually reaches Switch.+ *+ * This is not belt-and-braces. The Mattermost adapter delivers inbound posts off+ * the *agent bot's own websocket*, and that socket is opened on a background+ * thread after the bot account is created — the server log shows+ * `Websocket authentification OK` arriving after the room already exists. A+ * message posted in that window is silently dropped, which shows up later as a+ * scenario that times out for no visible reason.+ *+ * The probe is deliberately **unaddressed**: it proves Mattermost → Switch+ * delivery without putting an `@agent` mention on the wire, so it cannot spawn a+ * console session or consume a turn. It is read back through the agent's room+ * history, which carries ordinary chatter, rather than the notification stream,+ * which by design carries only addressed messages.+ */+export async function waitForBridgeReady(params: {+ switchClient: SwitchClient;+ mattermost: MattermostClient;+ agent: RegisteredAgent;+ room: RoomDetail;+ channel: MattermostChannel;+ runId: string;+ deadlineMs?: number;+}): Promise<void> {+ const deadlineMs = params.deadlineMs ?? 120_000;+ const probe = `switch-e2e-bridge-probe ${params.runId}`;+ const until = Date.now() + deadlineMs;++ while (Date.now() < until) {+ // Re-post rather than only re-read: if the socket was down when the last+ // one went out, that message is gone for good and no amount of polling+ // will produce it.+ await params.mattermost.post({ channelId: params.channel.id, message: probe });+ const history = await params.switchClient.roomHistory(params.agent, params.room.id, 50);+ if (history.some((event) => event.body.includes(probe))) return;+ await sleep(5_000);+ }++ throw new Error(+ `Mattermost -> Switch delivery never came up for room ${params.room.id} ` ++ `(channel ${params.channel.id}) within ${deadlineMs}ms: a plain channel message ` ++ `never reached the agent's room history. The agent bot's websocket is probably not connected.`+ );+}diff --git a/console/tooling-e2e/src/loopback.integration.test.ts b/console/tooling-e2e/src/loopback.integration.test.tsdeleted file mode 100644index 7228fdbc7..000000000--- a/console/tooling-e2e/src/loopback.integration.test.ts+++ /dev/null@@ -1,102 +0,0 @@-import { afterAll, beforeAll, describe, expect, it } from 'vitest';-import { describeEnv } from './env.ts';-import { resolveSkip, setupHarness, teardownHarness, type Harness } from './harness.ts';-import { eventText } from './switch-client.ts';--/**- * The harness's own plumbing, proved against a live stack with **no Switch- * Console and no model in the loop**: the harness plays both parts.- *- * 1. register an agent, create a channel, add the agent's bot to it, and confirm- * Switch provisions a room for that channel;- * 2. post `@agent …` as a human and confirm the addressed message reaches the- * agent-facing notification stream;- * 3. `POST /agents/{id}/message` as the agent and confirm it lands back in the- * Mattermost channel as the bot.- *- * If this file passes and `run.integration.test.ts` fails, the fault is in the- * console session, not in Switch, the bridge, or the harness.- */--let harness: Harness | null = null;-let skip: string | null = null;--beforeAll(async () => {- const resolved = await resolveSkip();- skip = resolved.skip;- if (skip || !resolved.env) return;- // auto_session off: nothing is expected to spawn a session — this test *is*- // the agent, so an auto_session profile would only invite the operator's- // console to fight it for the room.- harness = await setupHarness(resolved.env, { autoSession: false, prefix: 'e2e-loop' });- console.log(- `loopback: ${describeEnv(resolved.env)} agent=${harness.agent.name} (${harness.agent.id}) ` +- `channel=${harness.channel.name} (${harness.channel.id}) room=${harness.room.id}`- );-}, 300_000);--afterAll(async () => {- await teardownHarness(harness);-});--describe('Switch <-> Mattermost loopback', () => {- it('provisions a Switch room when the agent bot joins a Mattermost channel', () => {- if (skip) return void console.log(`SKIPPED: ${skip}`);- expect(harness).not.toBeNull();- expect(harness!.room.externalChannelId).toBe(harness!.channel.id);- expect(harness!.room.agentIds).toContain(harness!.agent.id);- });-- it('delivers an addressed channel message to the agent, and the agent reply back to the channel', async () => {- if (skip) return void console.log(`SKIPPED: ${skip}`);- const h = harness!;- const marker = `SWITCH_E2E_${h.runId.toUpperCase()}`;-- const sinceMs = Date.now();- const posted = await h.mattermost.post({- channelId: h.channel.id,- message: `@${h.agent.name} hello ${marker}`,- });- console.log(`loopback: posted ${posted.id} as ${h.human.username}`);-- // ── inbound: room -> Switch -> agent ────────────────────────────────────- const inbound = await h.switch.waitForNotification(- h.agent,- (event) => eventText(event).includes(marker),- 120_000- );- if (!inbound.match) {- throw new Error(- `Addressed message never reached the agent's notification stream. Saw: ${JSON.stringify(- inbound.seen- ).slice(0, 1000)}`- );- }- expect(inbound.match.room_id).toBe(h.room.id);- console.log(- `loopback: agent received event type=${inbound.match.type} room=${inbound.match.room_id}`- );-- // ── outbound: agent -> Switch -> room ───────────────────────────────────- const reply = `ack ${marker}`;- const eventId = await h.switch.sendMessage(h.agent, h.room.id, reply);- expect(eventId).toBeTruthy();-- const outbound = await h.mattermost.waitForPost({- channelId: h.channel.id,- fromUserId: h.bot.id,- predicate: (post) => post.message.includes(marker),- sinceMs,- deadlineMs: 120_000,- });- if (!outbound.match) {- throw new Error(- `Agent reply never appeared in Mattermost as bot ${h.bot.username}. Transcript: ` +- outbound.transcript.map((p) => `${p.user_id}: ${p.message}`).join(' | ')- );- }- console.log(- `loopback: bot post ${outbound.match.id} (matrix event ${eventId}) — round trip closed`- );- }, 400_000);-});diff --git a/console/tooling-e2e/src/console-credentials.ts b/console/tooling-e2e/src/provider/console-credentials.tssimilarity index 98%rename from console/tooling-e2e/src/console-credentials.tsrename to console/tooling-e2e/src/provider/console-credentials.tsindex b4ba8b353..b9f2173de 100644--- a/console/tooling-e2e/src/console-credentials.ts+++ b/console/tooling-e2e/src/provider/console-credentials.ts@@ -1,6 +1,6 @@import { promises as fs } from 'node:fs';import path from 'node:path';-import type { RegisteredAgent } from './switch-client.ts';+import type { RegisteredAgent } from '../clients/switch-client.ts';/*** Write the harness's throwaway agent into a working directory the way Switchdiff --git a/console/tooling-e2e/src/harness.ts b/console/tooling-e2e/src/provider/harness.tssimilarity index 55%rename from console/tooling-e2e/src/harness.tsrename to console/tooling-e2e/src/provider/harness.tsindex 25919ab71..17269c4bf 100644--- a/console/tooling-e2e/src/harness.ts+++ b/console/tooling-e2e/src/provider/harness.ts@@ -1,18 +1,25 @@import { promises as fs } from 'node:fs';+import type {+ MattermostChannel,+ MattermostClient,+ MattermostUser,+} from '../clients/mattermost-client.ts';+import {+ sleep,+ type RegisteredAgent,+ type RoomDetail,+ type SwitchClient,+} from '../clients/switch-client.ts';+import type { HarnessAgentType, HarnessEnv } from '../core/env.ts';+import { RunContext } from '../core/run.ts';+import type { LaneTarget } from '../core/suite.ts';+import { waitForBridgeReady } from '../fixtures/fixtures.ts';import {removeAgentCredentials,removeClaudeAgentDefinition,writeAgentCredentials,writeClaudeAgentDefinition,} from './console-credentials.ts';-import { loadEnv, shortId, type HarnessAgentType, type HarnessEnv } from './env.ts';-import {- isValidMattermostUsername,- MattermostClient,- type MattermostChannel,- type MattermostUser,-} from './mattermost-client.ts';-import { sleep, SwitchClient, type RegisteredAgent, type RoomDetail } from './switch-client.ts';/*** Everything a scenario needs: the two clients, the throwaway agent, the@@ -35,142 +42,75 @@ export interface Harness {credentialsPath?: string;}-export interface SetupOptions {- /**- * `false` registers the agent with `auto_session` off, so Switch will not- * expect a connector to spawn a session — used by the loopback check, which- * plays the agent itself.- */- autoSession?: boolean;- /** Prefix for the agent name and channel name. Must stay short (see below). */- prefix?: string;-}--/** The name prefix per agent type, kept inside Mattermost's 22-character cap. */-const PREFIX_BY_AGENT_TYPE: Record<HarnessAgentType, string> = {- opencode: 'e2e-opencode',- antigravity: 'e2e-antigravity',- cursor: 'e2e-cursor',- 'claude-code': 'e2e-claude',- codex: 'e2e-codex',-};--/**- * Why the name is built this way: the agent name becomes a **Mattermost bot- * username** verbatim, and Mattermost caps usernames at 22 characters and allows- * only `[a-z0-9._-]`. `e2e-opencode-` (13) + a 6-character run id fits with room- * to spare; anything longer silently fails bot creation on the bridge side and- * the room is then never provisioned.- */-export function agentNameFor(runId: string, prefix = PREFIX_BY_AGENT_TYPE.opencode): string {- const name = `${prefix}-${runId}`;- if (!isValidMattermostUsername(name)) {- throw new Error(- `Derived agent name '${name}' is not a valid Mattermost bot username ` +- `(lowercase, 3-22 chars, [a-z0-9._-]). Shorten the prefix.`- );- }- return name;-}-/**- * Reasons the suite skips rather than fails. A skip is only ever "this machine- * isn't set up", never "the thing under test is broken".+ * The agent-name role per agent type. The full name is+ * `e2e-<runId><scope>-<role>` and becomes a Mattermost bot username, which caps+ * at 22 characters — so a role can be at most ten.*/-export type SkipReason = string;--export async function resolveSkip(): Promise<{ skip: SkipReason | null; env: HarnessEnv | null }> {- if (process.env.SWITCH_E2E !== '1') {- return { skip: 'SWITCH_E2E=1 is not set (this suite drives a real Switch server)', env: null };- }-- let env: HarnessEnv;- try {- env = loadEnv();- } catch (error) {- return { skip: error instanceof Error ? error.message : String(error), env: null };- }-- const client = new SwitchClient({- apiUrl: env.switchApiUrl,- registrationToken: env.agentRegistrationToken,- gatewayAdminEmail: env.gatewayAdminEmail,- gatewayAdminPassword: env.gatewayAdminPassword,- });- try {- await client.health();- } catch (error) {- return {- skip: `Switch server at ${env.switchApiUrl} is unreachable: ${- error instanceof Error ? error.message : String(error)- }`,- env: null,- };- }-- const mattermost = new MattermostClient({ url: env.mattermostUrl, token: env.mattermostToken });- try {- await mattermost.me();- } catch (error) {- return {- skip: `Mattermost at ${env.mattermostUrl} is unreachable or MATTERMOST_TOKEN is invalid: ${- error instanceof Error ? error.message : String(error)- }`,- env: null,- };- }-- return { skip: null, env };-}+const ROLE_BY_AGENT_TYPE: Record<HarnessAgentType, string> = {+ opencode: 'opencode',+ antigravity: 'antigrav',+ cursor: 'cursor',+ 'claude-code': 'claude',+ codex: 'codex',+};/**- * Register a throwaway agent, create a channel, invite the agent's bot into it,- * and wait for Switch to provision the room.+ * Register a throwaway agent, create a room on the Mattermost bridge for it,+ * and wait for the bridge to carry a message.** Order matters and is not interchangeable: the bot account only exists once the* agent is registered (registration is what calls `create_agent_identity` on- * every collaboration bridge), and the Switch room only exists once that bot is- * a channel member.+ * every collaboration bridge), and the room is created with that bot in it.+ *+ * Unlike the fast lane's fixtures this setup outlives a single test: the+ * provider scenarios share one agent and one channel, and the agent may have+ * been registered by an earlier process (see {@link HarnessManifest}).*/-export async function setupHarness(env: HarnessEnv, options: SetupOptions = {}): Promise<Harness> {- const reused = await readManifest(env.manifestPath);- if (reused) return reuseHarness(env, reused);-- const runId = shortId();- const agentName = agentNameFor(runId, options.prefix ?? PREFIX_BY_AGENT_TYPE[env.agentType]);+export async function setupHarness(target: LaneTarget): Promise<Harness> {+ const { env, switchClient } = target;+ const mattermost = target.mattermost;+ if (!mattermost || !env.mattermost) {+ throw new Error(+ 'The provider lane needs Mattermost (MATTERMOST_URL, MATTERMOST_TOKEN, MATTERMOST_TEAM).'+ );+ }+ const reused = await readManifest(env.provider.manifestPath);+ if (reused) return reuseHarness(target, mattermost, reused);- const switchClient = new SwitchClient({- apiUrl: env.switchApiUrl,- registrationToken: env.agentRegistrationToken,- gatewayAdminEmail: env.gatewayAdminEmail,- gatewayAdminPassword: env.gatewayAdminPassword,- });- const mattermost = new MattermostClient({ url: env.mattermostUrl, token: env.mattermostToken });+ const run = new RunContext();+ const runId = run.runId;+ const names = run.scope();+ const agentName = names.name(ROLE_BY_AGENT_TYPE[env.provider.agentType]);const human = await mattermost.me();const bridgeId = await switchClient.defaultBridgeId('mattermost');// Resolved even though the room-create path does not need it: it fails fast// and clearly when MATTERMOST_TEAM names a team that does not exist.- await mattermost.findTeam(env.mattermostTeam);+ await mattermost.findTeam(env.mattermost.team);// `ClaudeCodeOptions` and `OpencodeOptions` agree on the two keys used here// (`auto_session`, `repo_dir`), so the payload is the same for both types.const agent = await switchClient.registerKnownAgent({- agentType: env.agentType,+ agentType: env.provider.agentType,name: agentName,description: `Switch Console end-to-end harness agent (run ${runId}). Safe to delete.`,options: {- auto_session: options.autoSession ?? true,- ...(env.agentRepoDir ? { repo_dir: env.agentRepoDir } : {}),+ auto_session: true,+ ...(env.provider.agentRepoDir ? { repo_dir: env.provider.agentRepoDir } : {}),},});+ // Owner-only by default, and the human the harness posts as is an unclaimed+ // Mattermost identity, so its messages would be refused rather than delivered.+ await switchClient.openAddressing(agent.id);+// Registration is what mints the bot; wait for it so a later `@name` mention// has something to resolve to.const bot = await mattermost.waitForBotUser(agent.name, 60_000);const room = await switchClient.createRoom({- name: `e2e-${runId}`,+ name: names.name('room'),description: `Switch Console end-to-end harness (run ${runId}). Safe to delete.`,bridgeId,agentNames: [agent.name],@@ -200,17 +140,17 @@ export async function setupHarness(env: HarnessEnv, options: SetupOptions = {}):// With SWITCH_E2E_AGENT_DIR set, provision the agent's identity into that// working directory so a session started there IS this agent. Without it the// operator has to add the agent in Switch Console by hand before the run.- if (env.agentRepoDir) {+ if (env.provider.agentRepoDir) {harness.credentialsPath = await writeAgentCredentials({- workingDir: env.agentRepoDir,+ workingDir: env.provider.agentRepoDir,agent,- apiEndpoint: env.switchApiUrl,+ apiEndpoint: env.switch.apiUrl,});// A Claude Code session runs *as* a named definition, so the definition has// to be on disk before the console starts one — see the writer's docblock.- if (env.agentType === 'claude-code') {+ if (env.provider.agentType === 'claude-code') {await writeClaudeAgentDefinition({- workingDir: env.agentRepoDir,+ workingDir: env.provider.agentRepoDir,agentName: agent.name,description: `Switch Console end-to-end harness agent (run ${runId}).`,// "Do exactly what you are asked" is load-bearing, and so is the second@@ -230,8 +170,8 @@ export async function setupHarness(env: HarnessEnv, options: SetupOptions = {}):}}- await waitForBridgeReady(harness);- await writeManifest(env.manifestPath, harness);+ await waitForBridgeReady({ switchClient, mattermost, agent, room, channel, runId });+ await writeManifest(env.provider.manifestPath, harness);return harness;}@@ -277,15 +217,12 @@ async function readManifest(path: string | null): Promise<HarnessManifest | null* bridge-readiness probe still runs: this process has proved nothing about the* bot's websocket, and reusing an agent is not evidence its socket is up now.*/-async function reuseHarness(env: HarnessEnv, manifest: HarnessManifest): Promise<Harness> {- const switchClient = new SwitchClient({- apiUrl: env.switchApiUrl,- registrationToken: env.agentRegistrationToken,- gatewayAdminEmail: env.gatewayAdminEmail,- gatewayAdminPassword: env.gatewayAdminPassword,- });- const mattermost = new MattermostClient({ url: env.mattermostUrl, token: env.mattermostToken });-+async function reuseHarness(+ target: LaneTarget,+ mattermost: MattermostClient,+ manifest: HarnessManifest+): Promise<Harness> {+ const { env, switchClient } = target;const harness: Harness = {env,switch: switchClient,@@ -298,53 +235,17 @@ async function reuseHarness(env: HarnessEnv, manifest: HarnessManifest): PromiserunId: manifest.runId,};- await waitForBridgeReady(harness);+ await waitForBridgeReady({+ switchClient,+ mattermost,+ agent: harness.agent,+ room: harness.room,+ channel: harness.channel,+ runId: harness.runId,+ });return harness;}-/**- * Block until a plain channel message actually reaches Switch.- *- * This is not belt-and-braces. The Mattermost adapter delivers inbound posts off- * the *agent bot's own websocket*, and that socket is opened on a background- * thread after the bot account is created — the server log shows- * `Websocket authentification OK` arriving after the room already exists. A- * message posted in that window is silently dropped, which shows up later as a- * scenario that times out for no visible reason.- *- * The probe is deliberately **unaddressed**: it proves Mattermost → Switch- * delivery without putting an `@agent` mention on the wire, so it cannot spawn a- * console session or consume a turn. It is read back through the agent's room- * history, which carries ordinary chatter, rather than the notification stream,- * which by design carries only addressed messages.- */-export async function waitForBridgeReady(harness: Harness, deadlineMs = 120_000): Promise<void> {- const probe = `switch-e2e-bridge-probe ${harness.runId}`;- const until = Date.now() + deadlineMs;- let posted = false;-- while (Date.now() < until) {- if (!posted) {- await harness.mattermost.post({ channelId: harness.channel.id, message: probe });- posted = true;- }- const history = await harness.switch.roomHistory(harness.agent, harness.room.id, 50);- if (history.some((event) => event.body.includes(probe))) return;-- // Re-post rather than only re-read: if the socket was down when the first- // one went out, that message is gone for good and no amount of polling- // will produce it.- await sleep(5_000);- posted = false;- }-- throw new Error(- `Mattermost -> Switch delivery never came up for room ${harness.room.id} ` +- `(channel ${harness.channel.id}) within ${deadlineMs}ms: a plain channel message ` +- `never reached the agent's room history. The agent bot's websocket is probably not connected.`- );-}-/** Statuses that mean a session is attending — or is about to be spawned for — the room. */const READY_STATUSES = new Set(['live', 'dormant']);@@ -392,17 +293,17 @@ export async function teardownHarness(// Before anything is deleted: a manifest that outlives its agent sends the// next run at a room that is gone.- if (harness.env.manifestPath) {- await fs.rm(harness.env.manifestPath, { force: true }).catch(() => undefined);+ if (harness.env.provider.manifestPath) {+ await fs.rm(harness.env.provider.manifestPath, { force: true }).catch(() => undefined);}- if (harness.env.agentRepoDir) {+ if (harness.env.provider.agentRepoDir) {await removeAgentCredentials({- workingDir: harness.env.agentRepoDir,+ workingDir: harness.env.provider.agentRepoDir,agentName: harness.agent.name,}).catch(() => undefined);await removeClaudeAgentDefinition({- workingDir: harness.env.agentRepoDir,+ workingDir: harness.env.provider.agentRepoDir,agentName: harness.agent.name,}).catch(() => undefined);}diff --git a/console/tooling-e2e/src/provider/provider.e2e.test.ts b/console/tooling-e2e/src/provider/provider.e2e.test.tsnew file mode 100644index 000000000..76f93e93c--- /dev/null+++ b/console/tooling-e2e/src/provider/provider.e2e.test.ts@@ -0,0 +1,86 @@+import { afterAll, beforeAll, describe, expect, it } from 'vitest';+import { describeEnv } from '../core/env.ts';+import { admit, assertValidHeaders } from '../core/scenario.ts';+import { resolveLane, type LaneResolution } from '../core/suite.ts';+import { setupHarness, teardownHarness, waitForSession, type Harness } from './harness.ts';+import { formatResultsTable, SCENARIOS, type ScenarioResult } from './scenarios.ts';++/**+ * The provider lane. One throwaway agent, one Mattermost channel, the scenarios+ * run in order against them, then teardown and a summary table. Runs when+ * `SWITCH_E2E_LANE=provider`.+ *+ * A failure only means something when a real agent session is attending the+ * room: a Switch Console session, or Console's auto-session watcher, for the+ * agent this setup registers — see README.md, "What must be running".+ */++assertValidHeaders(SCENARIOS.map((scenario) => scenario.header));++let lane: LaneResolution;+let harness: Harness | null = null;+const results: ScenarioResult[] = [];++beforeAll(async () => {+ lane = await resolveLane('provider');+ if (!lane.target) return;+ // Nothing to set up when no scenario can run here — say, a target that+ // lacks the prerequisites every scenario shares.+ if (!SCENARIOS.some((scenario) => admit(scenario.header, lane.target!.env).run)) return;++ harness = await setupHarness(lane.target);+ console.log(+ [+ '',+ `Provider lane run ${harness.runId}`,+ ` ${describeEnv(lane.target.env)}`,+ ` agent ${harness.agent.name} (${harness.agent.id})`,+ ` bot @${harness.bot.username} (${harness.bot.id})`,+ ` channel ${harness.channel.name} (${harness.channel.id})`,+ ` room ${harness.room.name} (${harness.room.id})`,+ '',+ ` A Switch Console session must now be running for '${harness.agent.name}'.`,+ '',+ ].join('\n')+ );++ const status = await waitForSession(+ harness,+ Number(process.env.SWITCH_E2E_SESSION_TIMEOUT_MS ?? 120_000)+ );+ console.log(` Switch reports the agent as '${status}' in the room — running scenarios.\n`);+}, 600_000);++afterAll(async () => {+ if (results.length > 0) console.log(formatResultsTable(results));+ await teardownHarness(harness);+});++describe('provider lane', () => {+ for (const scenario of SCENARIOS) {+ it(scenario.header.id, async (context) => {+ if (!lane.target) return context.skip(lane.skip);+ const admission = admit(scenario.header, lane.target.env);+ if (!admission.run) {+ if (admission.failure) {+ throw new Error(+ `${scenario.header.id} cannot run (SWITCH_E2E_STRICT=1): ${admission.reason}`+ );+ }+ return context.skip(admission.reason);+ }+ const h = harness!;+ const result = await scenario.run(h);+ results.push(result);+ if (!result.ok) {+ console.log(+ `${result.name} transcript:\n` ++ result.transcript+ .map((post) => ` [${post.user_id === h.bot.id ? 'bot ' : 'user'}] ${post.message}`)+ .join('\n')+ );+ }+ expect(result.ok, result.details).toBe(true);+ });+ }+});diff --git a/console/tooling-e2e/src/scenarios.test.ts b/console/tooling-e2e/src/provider/scenarios.test.tssimilarity index 100%rename from console/tooling-e2e/src/scenarios.test.tsrename to console/tooling-e2e/src/provider/scenarios.test.tsdiff --git a/console/tooling-e2e/src/scenarios.ts b/console/tooling-e2e/src/provider/scenarios.tssimilarity index 91%rename from console/tooling-e2e/src/scenarios.tsrename to console/tooling-e2e/src/provider/scenarios.tsindex 1a54be982..eeb6b4d5f 100644--- a/console/tooling-e2e/src/scenarios.ts+++ b/console/tooling-e2e/src/provider/scenarios.ts@@ -1,7 +1,8 @@import { readFile } from 'node:fs/promises';import path from 'node:path';+import { sleep, type MattermostPost } from '../clients/mattermost-client.ts';+import type { ScenarioHeader } from '../core/scenario.ts';import type { Harness } from './harness.ts';-import { sleep, type MattermostPost } from './mattermost-client.ts';/*** The scenarios. Each one drives the full path@@ -218,7 +219,7 @@ const CHOSEN = 'green';*/export const question: Scenario = (harness) =>scenario('question', async (record) => {- const prose = harness.env.questionMode === 'prose';+ const prose = harness.env.provider.questionMode === 'prose';const since = await ask(harness,prose@@ -322,7 +323,7 @@ async function markerContents(file: string): Promise<string | null> {/** A real local command must start and then be cancelled before its final write. */export const interrupt: Scenario = (harness) =>scenario('interrupt', async (record) => {- const dir = harness.env.agentRepoDir;+ const dir = harness.env.provider.agentRepoDir;if (!dir)throw new Error('The interrupt scenario requires SWITCH_E2E_AGENT_DIR on this machine.');const stem = `interrupt-${harness.runId}-${Date.now()}`;@@ -365,7 +366,47 @@ export const interrupt: Scenario = (harness) =>return 'command start confirmed; completion marker absent after 95s';});-export const SCENARIOS: Scenario[] = [greet, question, approval, interrupt];+/**+ * A provider-lane scenario: its header, and the function that drives it. All of+ * them share one agent and one channel, set up once per run.+ */+export interface ProviderScenario {+ header: ScenarioHeader;+ run: Scenario;+}++/** Writes to a room, so never prod. */+const PROVIDER_ENVIRONMENTS: ScenarioHeader['environments'] = [+ 'ephemeral',+ 'local',+ 'dev',+ 'staging',+];++function providerHeader(+ id: string,+ covers: string[],+ requires: ScenarioHeader['requires']+): ScenarioHeader {+ return {+ id,+ covers,+ needsProvider: true,+ environments: PROVIDER_ENVIRONMENTS,+ owner: 'console',+ requires: ['mattermost', ...requires],+ };+}++export const SCENARIOS: readonly ProviderScenario[] = [+ { header: providerHeader('greet', ['connect_to_room', 'post_message'], []), run: greet },+ { header: providerHeader('question', ['connect_to_room', 'post_message'], []), run: question },+ { header: providerHeader('approval', ['connect_to_room', 'post_message'], []), run: approval },+ {+ header: providerHeader('interrupt', ['connect_to_room', 'post_message'], ['agent-dir']),+ run: interrupt,+ },+];/** Fixed-width summary of a run, printed at the end of the suite. */export function formatResultsTable(results: ScenarioResult[]): string {diff --git a/console/tooling-e2e/src/seed.ts b/console/tooling-e2e/src/provider/seed.tssimilarity index 76%rename from console/tooling-e2e/src/seed.tsrename to console/tooling-e2e/src/provider/seed.tsindex 53421f464..8977efe59 100644--- a/console/tooling-e2e/src/seed.ts+++ b/console/tooling-e2e/src/provider/seed.ts@@ -9,10 +9,10 @@* has to be up before the first scenario posts. Splitting setup out gives the* three steps somewhere to happen in:*- * SWITCH_E2E=1 SWITCH_E2E_MANIFEST=… SWITCH_E2E_AGENT_DIR=… \- * node --experimental-strip-types src/seed.ts+ * SWITCH_E2E_LANE=provider SWITCH_E2E_MANIFEST=… SWITCH_E2E_AGENT_DIR=… \+ * node --experimental-strip-types src/provider/seed.ts* # seed the console's database from the manifest, start the console- * SWITCH_E2E=1 SWITCH_E2E_MANIFEST=… … vitest run src/run.integration.test.ts+ * SWITCH_E2E_LANE=provider SWITCH_E2E_MANIFEST=… … vitest run src/provider** The scenario run finds the manifest, reuses what is in it rather than* registering a second agent, and deletes it during teardown.@@ -26,23 +26,25 @@* `provider_id` `'opencode'` / `'claude'` and a `provider_config` of* `{"version":"2","providerId":"<that>","values":{},"runtime":"provider"}`.*/-import { loadEnv } from './env.ts';+import { resolveLane } from '../core/suite.ts';import { setupHarness } from './harness.ts';export async function seed(log: (message: string) => void = console.log): Promise<void> {- const env = loadEnv();- if (!env.manifestPath) {+ const lane = await resolveLane('provider');+ if (!lane.target) throw new Error(`Cannot seed: ${lane.skip}`);+ const env = lane.target.env;+ if (!env.provider.manifestPath) {throw new Error('SWITCH_E2E_MANIFEST must name the file to record the agent and room in.');}- const harness = await setupHarness(env);+ const harness = await setupHarness(lane.target);log([- `type ${env.agentType} (question mode: ${env.questionMode})`,+ `type ${env.provider.agentType} (question mode: ${env.provider.questionMode})`,`agent ${harness.agent.name} (${harness.agent.id})`,`bot @${harness.bot.username} (${harness.bot.id})`,`channel ${harness.channel.name} (${harness.channel.id})`,`room ${harness.room.name} (${harness.room.id})`,- `manifest ${env.manifestPath}`,+ `manifest ${env.provider.manifestPath}`,].join('\n'));}diff --git a/console/tooling-e2e/src/run.integration.test.ts b/console/tooling-e2e/src/run.integration.test.tsdeleted file mode 100644index 730d7ce19..000000000--- a/console/tooling-e2e/src/run.integration.test.ts+++ /dev/null@@ -1,84 +0,0 @@-import { afterAll, beforeAll, describe, expect, it } from 'vitest';-import { describeEnv } from './env.ts';-import {- resolveSkip,- setupHarness,- teardownHarness,- waitForSession,- type Harness,-} from './harness.ts';-import { formatResultsTable, SCENARIOS, type ScenarioResult } from './scenarios.ts';--/**- * The suite. One throwaway agent, one Mattermost channel, four scenarios run in- * order against them, then teardown and a summary table.- *- * It **skips** — never fails — when the machine is not set up for it:- * `SWITCH_E2E=1` unset, configuration missing, or Switch/Mattermost unreachable.- * The reason is printed, so a skip is never mysterious.- *- * It **fails** when the stack is there and the agent does not behave. For a- * failure to mean anything, a Switch Console session must be running for the- * registered agent — see README.md, "What must be running".- */--let harness: Harness | null = null;-let skip: string | null = null;-const results: ScenarioResult[] = [];--beforeAll(async () => {- const resolved = await resolveSkip();- skip = resolved.skip;- if (skip || !resolved.env) return;-- harness = await setupHarness(resolved.env);- console.log(- [- '',- `Switch Console end-to-end run ${harness.runId}`,- ` ${describeEnv(resolved.env)}`,- ` agent ${harness.agent.name} (${harness.agent.id})`,- ` bot @${harness.bot.username} (${harness.bot.id})`,- ` channel ${harness.channel.name} (${harness.channel.id})`,- ` room ${harness.room.name} (${harness.room.id})`,- '',- ` A Switch Console session must now be running for '${harness.agent.name}'.`,- '',- ].join('\n')- );-- const status = await waitForSession(- harness,- Number(process.env.SWITCH_E2E_SESSION_TIMEOUT_MS ?? 120_000)- );- console.log(` Switch reports the agent as '${status}' in the room — running scenarios.\n`);-}, 600_000);--afterAll(async () => {- if (results.length > 0) console.log(formatResultsTable(results));- await teardownHarness(harness);-});--describe('Switch Console provider-backed session, end to end', () => {- for (const runScenario of SCENARIOS) {- // The scenario functions are named after the behaviour they exercise, so the- // test name comes from the function rather than being restated here.- it(runScenario.name, async () => {- if (skip) {- console.log(`SKIPPED (${runScenario.name}): ${skip}`);- return;- }- const result = await runScenario(harness!);- results.push(result);- if (!result.ok) {- console.log(- `${result.name} transcript:\n` +- result.transcript- .map((post) => ` [${post.user_id === harness!.bot.id ? 'bot ' : 'user'}] ${post.message}`)- .join('\n')- );- }- expect(result.ok, result.details).toBe(true);- });- }-});diff --git a/console/tooling-e2e/vitest.config.ts b/console/tooling-e2e/vitest.config.tsindex 16c4d985e..98fb90585 100644--- a/console/tooling-e2e/vitest.config.ts+++ b/console/tooling-e2e/vitest.config.ts@@ -3,11 +3,12 @@ import { defineConfig } from 'vitest/config';export default defineConfig({test: {include: ['src/**/*.test.ts'],- // Real network round-trips through Switch, Matrix and Mattermost; a whole+ // Real network round-trips through Switch and Mattermost; a whole// scenario suite can idle for minutes waiting on a model turn.testTimeout: 15 * 60_000,hookTimeout: 5 * 60_000,- // Scenarios share one channel and one agent, so they must not interleave.+ // Provider scenarios share one channel and one agent, and a fast-lane run+ // shares one run id, so nothing may interleave.fileParallelism: false,pool: 'forks',reporters: ['verbose'],diff --git a/justfile b/justfileindex f053838f2..6fcbd3ccf 100644--- a/justfile+++ b/justfile@@ -171,6 +171,14 @@ test *args:e2e-follow-ups *args:core/tests/e2e/follow_the_conversation/run.sh {{ args }}+# ── End-to-end harness, fast lane, on a throwaway stack ────────────────────────+# Fresh Postgres and Mattermost, switch-core from this checkout, then the+# provider-free scenarios in console/tooling-e2e, then teardown. Needs Docker+# and your local stack stopped. `--keep` leaves it running; arguments after+# `--` go to vitest. Output: build/e2e/ephemeral.+e2e-fast *args:+ console/tooling-e2e/scripts/ephemeral-stack.sh {{ args }}+# ── Run integration tests (real Postgres via testcontainers) ───────────────────# DOCKER_HOST is auto-resolved from the active docker context in conftest, so this# works under Docker Desktop / OrbStack / colima without extra setup.