Skip to content

Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.

Address review of the explore lane

Petr Bauchdda0c6fb1d5b
diff --git a/console/tooling-e2e/README.md b/console/tooling-e2e/README.md
index 35b3f17c6..def07250a 100644
--- a/console/tooling-e2e/README.md
+++ b/console/tooling-e2e/README.md
@@ -214,8 +214,9 @@ beside this file. Never commit one. `just e2e-fast` sets all of them itself.
| `SWITCH_E2E_KEEP` | no | `1` leaves agents, rooms and channels behind for inspection. |
| `SWITCH_E2E_SERVER_LOG` | no | The server's log, written with `LOG_FORMAT=json`. When set, a scenario fails if the server logged an error while it ran. The stack script sets it. |
| `SWITCH_E2E_SECRETS_FILE` | no | File to append each credential the server issues during the run to (agent keys, gateway sessions), for CI's scan of the output. The stack script sets it. |
-| `SWITCH_GATEWAY_UI_URL` | browser lane | Where the gateway UI is served. Its `/gateway` requests must reach `SWITCH_API_URL`. `just e2e-browser` sets it. |
-| `SWITCH_E2E_OUTPUT_DIR` | no | Where the browser lane saves screenshots of failed pages. The stack script sets it. |
+| `SWITCH_GATEWAY_UI_URL` | browser and explore lanes | Where the gateway UI is served. Its `/gateway` requests must reach `SWITCH_API_URL`. `just e2e-browser` and `just e2e-explore` set it. |
+| `SWITCH_E2E_OUTPUT_DIR` | explore lane | Where the browser lane saves screenshots of failed pages, and the explore lane its trace (in `explore/`). The stack script sets it. |
+| `SWITCH_E2E_EXPLORE_TIME_LIMIT` | no | How long the explore lane explores, e.g. `90s`, `5m` (the default), `1h`. Read only by that lane. |
| `SWITCH_E2E_DRIVER_MODEL` | agentic lane | Exact model id that plays the human, e.g. `claude-sonnet-5`. |
| `SWITCH_E2E_JUDGE_MODEL` | agentic lane | Exact model id that grades the conversation. |
| `SWITCH_E2E_AGENTIC_BUDGET_USD` | agentic lane | What one run may spend, driver, judge and agent together. The run stops once it is reached. |
@@ -401,7 +402,8 @@ the browser lane, the server log must stay free of errors for the whole run.
The explorer deletes, archives and renames whatever it finds, so the lane
refuses to run on anything but an ephemeral stack. It cannot sign out, since
-that would end the session it is testing, and it acts only once signed in.
+that would end the session it is testing, nor click Change password, the entry
+next to Sign out, and it acts only once signed in.
Everything else is fair game, including deleting things behind a confirmation
dialog. The lane seeds an agent and two rooms first, so the pages have
something on them.
diff --git a/console/tooling-e2e/src/core/env.ts b/console/tooling-e2e/src/core/env.ts
index 23dfac5b4..f64665d49 100644
--- a/console/tooling-e2e/src/core/env.ts
+++ b/console/tooling-e2e/src/core/env.ts
@@ -142,11 +142,6 @@ export interface HarnessEnv {
* explore lane its trace, if anywhere.
*/
outputDir: string | null;
- /**
- * How long the explore lane explores, as Bombadil's `--time-limit` takes it
- * (`90s`, `5m`, `1h`). `SWITCH_E2E_EXPLORE_TIME_LIMIT`, default `5m`.
- */
- exploreTimeLimit: string;
switch: SwitchEnv;
/** `null` when no Mattermost is configured; scenarios that need one then skip or fail. */
mattermost: MattermostEnv | null;
@@ -247,9 +242,10 @@ export function strictFrom(
}
/**
- * The explore lane's time limit. Read on its own as well as by `loadEnv`,
- * because the lane sizes its case's timeout from it before the rest of the
- * configuration is loaded.
+ * How long the explore lane explores, as Bombadil's `--time-limit` takes it
+ * (`90s`, `5m`, `1h`): `SWITCH_E2E_EXPLORE_TIME_LIMIT`, default `5m`. Not part of
+ * `loadEnv`, so a malformed value fails only the explore lane, which reads it
+ * when it collects its cases, to size the exploring case's timeout.
*/
export function exploreTimeLimitFrom(
source: Source = process.env,
@@ -325,7 +321,6 @@ export function loadEnv(
serverLog: get('SWITCH_E2E_SERVER_LOG') ?? null,
gatewayUiUrl: optionalUrl(get('SWITCH_GATEWAY_UI_URL')),
outputDir: get('SWITCH_E2E_OUTPUT_DIR') ?? null,
- exploreTimeLimit: exploreTimeLimitFrom(source, file),
switch: {
apiUrl: stripTrailingSlash(required.SWITCH_API_URL as string),
gatewayAdminEmail: required.SWITCH_GATEWAY_ADMIN_EMAIL as string,
diff --git a/console/tooling-e2e/src/explore/explore.e2e.test.ts b/console/tooling-e2e/src/explore/explore.e2e.test.ts
index 947860aa5..d9d826517 100644
--- a/console/tooling-e2e/src/explore/explore.e2e.test.ts
+++ b/console/tooling-e2e/src/explore/explore.e2e.test.ts
@@ -1,4 +1,4 @@
-import { spawn } from 'node:child_process';
+import { execFileSync, spawn, type ChildProcess } from 'node:child_process';
import {
copyFileSync,
createWriteStream,
@@ -8,13 +8,21 @@ import {
rmSync,
writeFileSync,
} from 'node:fs';
+import { createRequire } from 'node:module';
import { tmpdir } from 'node:os';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { chromium } from 'playwright';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { SwitchHttpError, type SwitchClient } from '../clients/switch-client.ts';
-import { describeEnv, durationMs, exploreTimeLimitFrom, type HarnessEnv } from '../core/env.ts';
+import {
+ describeEnv,
+ durationMs,
+ exploreTimeLimitFrom,
+ laneFrom,
+ readDotEnvFile,
+ type HarnessEnv,
+} from '../core/env.ts';
import { RunContext, type NameScope } from '../core/run.ts';
import {
describeServerLogCheck,
@@ -57,9 +65,21 @@ const EXIT_VIOLATIONS = 2;
/** Over the time limit, what the explorer may take to start Chromium and write its trace. */
const SLACK_MS = 3 * 60_000;
+/** How long Bombadil has to stop on SIGINT, writing its trace, before it is killed. */
+const KILL_GRACE_MS = 10_000;
+
+/** The Bombadil binary the npm package ships for each platform it supports. */
+const BOMBADIL_BINARIES: Record<string, string> = {
+ 'linux-x64': 'bombadil-linux-x64',
+ 'linux-arm64': 'bombadil-linux-arm64',
+ 'darwin-arm64': 'bombadil-darwin-arm64',
+};
const run = new RunContext();
-const timeLimitMs = durationMs(exploreTimeLimitFrom());
+// Read only when this lane is asked for: a value the other lanes have no use
+// for must not fail them.
+const timeLimit =
+ laneFrom(process.env, readDotEnvFile()) === 'explore' ? exploreTimeLimitFrom() : null;
let lane: LaneResolution;
let watch: ServerLogWatch;
let fixtures: Fixtures | null = null;
@@ -68,6 +88,7 @@ let observer: ScenarioObserver | null = null;
let outputDir: string;
let trace: TraceSummary | null = null;
let serverErrors: ServerLogError[] | null = null;
+let explorer: ChildProcess | null = null;
beforeAll(async () => {
lane = await resolveLane('explore');
@@ -83,19 +104,22 @@ beforeAll(async () => {
if (env.gatewayUiUrl === null) {
throw new Error('The explore lane needs SWITCH_GATEWAY_UI_URL: where the gateway UI is served.');
}
+ if (env.outputDir === null) {
+ throw new Error(
+ 'The explore lane needs SWITCH_E2E_OUTPUT_DIR: where Bombadil writes the trace a failure is read from.'
+ );
+ }
console.log(`explore lane run ${run.runId}: ${describeEnv(env)}`);
console.log(` ${describeServerLogCheck(env.serverLog)}`);
watch = new ServerLogWatch(env.serverLog);
- outputDir =
- env.outputDir === null
- ? mkdtempSync(path.join(tmpdir(), 'switch-explore-'))
- : path.join(env.outputDir, 'explore');
+ outputDir = path.join(env.outputDir, 'explore');
names = run.scope();
fixtures = new Fixtures(env, names, switchClient, mattermost);
await seed(fixtures, switchClient, names);
}, 120_000);
afterAll(async () => {
+ if (explorer !== null) await stop(explorer);
observer?.stop();
if (fixtures === null) return;
const leftovers = await fixtures.teardown();
@@ -103,11 +127,12 @@ afterAll(async () => {
});
describe('explore lane', () => {
- it(`explores the gateway UI`, { timeout: timeLimitMs + SLACK_MS }, async (context) => {
- if (!lane.target) return context.skip(lane.skip);
+ const timeout = timeLimit === null ? undefined : durationMs(timeLimit) + SLACK_MS;
+ it(`explores the gateway UI`, { timeout }, async (context) => {
+ if (!lane.target || timeLimit === null) return context.skip(lane.skip ?? 'not asked for');
const { env, switchClient } = lane.target;
observer = watch.begin(switchClient, names.tag, 'explore');
- const exit = await explore(env, names.tag);
+ const exit = await explore(env, names.tag, timeLimit, context.signal);
serverErrors = observer.errors(null, (message) => console.log(`[explore] ${message}`));
observer.stop();
observer = null;
@@ -208,8 +233,17 @@ async function unlessGone(removal: Promise<void>): Promise<void> {
* Its output goes to a log file rather than this process's: it prints a line
* per action, thousands in a run. Every request the page makes carries the
* scope's tag, so an error the server logs serving one is attributed here.
+ *
+ * When the case times out or the run is cancelled (`signal`), Bombadil and the
+ * Chromium it started are stopped (see `stop`), rather than going on deleting
+ * things on the stack while teardown runs.
*/
-async function explore(env: HarnessEnv, tag: string): Promise<number> {
+async function explore(
+ env: HarnessEnv,
+ tag: string,
+ timeLimit: string,
+ signal: AbortSignal
+): Promise<number> {
rmSync(outputDir, { recursive: true, force: true });
mkdirSync(outputDir, { recursive: true });
const specDir = mkdtempSync(path.join(tmpdir(), 'switch-explore-spec-'));
@@ -224,14 +258,12 @@ async function explore(env: HarnessEnv, tag: string): Promise<number> {
{ mode: 0o600 }
);
const args = [
- 'exec',
- 'bombadil',
'browser',
'test',
'--headless',
'--no-sandbox',
'--time-limit',
- env.exploreTimeLimit,
+ timeLimit,
'--output-path',
outputDir,
'--width',
@@ -243,25 +275,126 @@ async function explore(env: HarnessEnv, tag: string): Promise<number> {
`${env.gatewayUiUrl}/login`,
path.join(specDir, 'gateway.ts'),
];
- console.log(`[explore] exploring for ${env.exploreTimeLimit}; Bombadil's log: ${logFile()}`);
+ console.log(`[explore] exploring for ${timeLimit}; Bombadil's log: ${logFile()}`);
const log = createWriteStream(logFile());
- const child = spawn('pnpm', args, {
+ const child = spawn(bombadilBinary(), args, {
// Bombadil finds no browser of its own; Playwright's is installed for this lane.
env: { ...process.env, CHROME: chromium.executablePath() },
stdio: ['ignore', 'pipe', 'pipe'],
+ detached: true,
});
+ explorer = child;
child.stdout.pipe(log);
child.stderr.pipe(log);
- const code = await new Promise<number>((resolve, reject) => {
- child.on('error', reject);
- child.on('close', (exitCode, signal) =>
- exitCode === null ? reject(new Error(`Bombadil was killed by ${signal}.`)) : resolve(exitCode)
- );
- });
- await new Promise((resolve) => log.end(resolve));
- return code;
+ const onAbort = () => void stop(child);
+ signal.addEventListener('abort', onAbort, { once: true });
+ const removeGuard = guardAgainstExit(child, specDir);
+ try {
+ return await new Promise<number>((resolve, reject) => {
+ child.on('error', reject);
+ child.on('close', (exitCode, killedBy) =>
+ exitCode === null
+ ? reject(new Error(`Bombadil was stopped by ${killedBy}.`))
+ : resolve(exitCode)
+ );
+ });
+ } finally {
+ signal.removeEventListener('abort', onAbort);
+ removeGuard();
+ explorer = null;
+ await new Promise((resolve) => log.end(resolve));
+ }
} finally {
rmSync(specDir, { recursive: true, force: true });
}
}
+const EXIT_SIGNALS = ['SIGINT', 'SIGTERM', 'SIGHUP'] as const;
+
+/**
+ * Kill Bombadil's process group and remove the spec copy (which holds the
+ * admin's password) if this worker goes away first: cancelling a run with
+ * Ctrl-C ends it without aborting the case's signal, and the group, detached
+ * so it can be stopped as one, would otherwise outlive it. Everything here is
+ * synchronous, as an `exit` handler must be. Returns the undo.
+ */
+function guardAgainstExit(child: ChildProcess, specDir: string): () => void {
+ const kill = () => {
+ killExplorer(child);
+ rmSync(specDir, { recursive: true, force: true });
+ };
+ const onSignal = (received: NodeJS.Signals) => {
+ kill();
+ process.kill(process.pid, received);
+ };
+ process.once('exit', kill);
+ process.once('disconnect', kill);
+ for (const name of EXIT_SIGNALS) process.once(name, onSignal);
+ return () => {
+ process.off('exit', kill);
+ process.off('disconnect', kill);
+ for (const name of EXIT_SIGNALS) process.off(name, onSignal);
+ };
+}
+
+/**
+ * Stop Bombadil the way Ctrl-C does (SIGINT, its only handled signal): it
+ * finishes, writes its trace and closes Chromium. Killed if it has not exited
+ * in time.
+ */
+async function stop(child: ChildProcess): Promise<void> {
+ if (child.pid === undefined || child.exitCode !== null || child.signalCode !== null) return;
+ const exited = new Promise((resolve) => child.once('close', resolve));
+ process.kill(child.pid, 'SIGINT');
+ const timer = new Promise((resolve) => setTimeout(resolve, KILL_GRACE_MS, 'timeout'));
+ if ((await Promise.race([exited, timer])) === 'timeout') {
+ console.log(`[explore] Bombadil did not stop within ${KILL_GRACE_MS / 1000} s; killing it.`);
+ killExplorer(child);
+ await exited;
+ }
+}
+
+/**
+ * SIGKILL Bombadil's process group and each Chromium's. Bombadil starts
+ * Chromium in a group of its own and closes it only on a clean exit, so a
+ * killed Bombadil would leave it running: its groups are found first, from
+ * Bombadil's children. Synchronous, for the exit guard.
+ */
+function killExplorer(child: ChildProcess): void {
+ if (child.pid === undefined || child.exitCode !== null || child.signalCode !== null) return;
+ for (const leader of [...childPids(child.pid), child.pid]) {
+ try {
+ process.kill(-leader, 'SIGKILL');
+ } catch (error) {
+ // A child that leads no group of its own went with Bombadil's.
+ if ((error as NodeJS.ErrnoException).code !== 'ESRCH') throw error;
+ }
+ }
+}
+
+function childPids(pid: number): number[] {
+ try {
+ return execFileSync('pgrep', ['-P', String(pid)], { encoding: 'utf8' })
+ .split('\n')
+ .filter((line) => line.trim() !== '')
+ .map(Number);
+ } catch (error) {
+ // pgrep exits 1 when nothing matches.
+ if ((error as { status?: number }).status === 1) return [];
+ throw error;
+ }
+}
+
+/** The binary the npm package ships for this platform, run without its Node launcher. */
+function bombadilBinary(): string {
+ const key = `${process.platform}-${process.arch}`;
+ const binary = BOMBADIL_BINARIES[key];
+ if (binary === undefined) throw new Error(`Bombadil ships no binary for ${key}.`);
+ // The package's exports name only its types, so look for it where Node would.
+ const searched = createRequire(import.meta.url).resolve.paths('@antithesishq/bombadil') ?? [];
+ for (const dir of searched) {
+ const candidate = path.join(dir, '@antithesishq', 'bombadil', 'binaries', binary);
+ if (existsSync(candidate)) return candidate;
+ }
+ throw new Error(`No ${binary} in @antithesishq/bombadil; is it installed (pnpm install)?`);
+}
diff --git a/console/tooling-e2e/src/explore/explore.test.ts b/console/tooling-e2e/src/explore/explore.test.ts
index fd77d534e..c7383c97b 100644
--- a/console/tooling-e2e/src/explore/explore.test.ts
+++ b/console/tooling-e2e/src/explore/explore.test.ts
@@ -2,6 +2,7 @@ import { readFileSync } from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { describe, expect, it } from 'vitest';
+import { durationMs, exploreTimeLimitFrom, loadEnv } from '../core/env.ts';
import { ACTION_GENERATORS, PROPERTIES } from './properties.ts';
import { describeViolation, summarizeTrace } from './trace.ts';
@@ -91,6 +92,28 @@ describe('explore lane', () => {
expect(describeViolation(violation)).toBe('() => b');
});
+ it('says an or failed on both sides', () => {
+ const side = (condition: string) => ({ False: { time: 1, condition, snapshots: [] } });
+ expect(describeViolation({ Or: { left: side('() => a'), right: side('() => b') } })).toBe(
+ 'neither () => a nor () => b'
+ );
+ });
+
+ it('reads its time limit apart from the other lanes', () => {
+ const malformed = { SWITCH_E2E_EXPLORE_TIME_LIMIT: '1m30s' };
+ const base = {
+ SWITCH_API_URL: 'http://switch.invalid',
+ SWITCH_GATEWAY_ADMIN_EMAIL: '[email protected]',
+ SWITCH_GATEWAY_ADMIN_PASSWORD: 'placeholder',
+ SWITCH_AGENT_REGISTRATION_TOKEN: 'placeholder',
+ SWITCH_E2E_ENVIRONMENT: 'ephemeral',
+ };
+ expect(() => loadEnv('fast', { ...base, ...malformed }, {})).not.toThrow();
+ expect(() => exploreTimeLimitFrom(malformed, {})).toThrow(/whole number with a unit/);
+ expect(exploreTimeLimitFrom({}, {})).toBe('5m');
+ expect(durationMs('90s')).toBe(90_000);
+ });
+
it('refuses a trace line that is not JSON', () => {
expect(() => summarizeTrace('{"timestamp":')).toThrow(/trace line 1 is not JSON/);
});
diff --git a/console/tooling-e2e/src/explore/properties.ts b/console/tooling-e2e/src/explore/properties.ts
index ef5075300..9fb03a0b7 100644
--- a/console/tooling-e2e/src/explore/properties.ts
+++ b/console/tooling-e2e/src/explore/properties.ts
@@ -15,4 +15,4 @@ export const PROPERTIES = [
] as const;
/** The spec's other exports: its action generators. */
-export const ACTION_GENERATORS = ['signingIn', 'exploring', 'waiting'] as const;
+export const ACTION_GENERATORS = ['signingIn', 'settling', 'exploring', 'waiting'] as const;
diff --git a/console/tooling-e2e/src/explore/spec/gateway.ts b/console/tooling-e2e/src/explore/spec/gateway.ts
index eccaea755..d9b73bb55 100644
--- a/console/tooling-e2e/src/explore/spec/gateway.ts
+++ b/console/tooling-e2e/src/explore/spec/gateway.ts
@@ -51,11 +51,11 @@ const ROUTES = [
/**
* Signing out ends the session the explorer is testing, and signing back in
- * proves nothing the browser lane does not. (Changing the password needs the
- * current one, which random input does not know, so that dialog stays open to
- * it.)
+ * proves nothing the browser lane does not. Change password is the entry next
+ * to it, and the menu shifts while it settles, so a click aimed there can land
+ * on Sign out: the explorer clicks neither.
*/
-const SIGN_OUT = 'Sign out';
+const SESSION_ENDING = ['Sign out', 'Change password'];
const path = extract((state) => state.window.location.pathname);
@@ -76,8 +76,10 @@ const signedIn = extract(
/**
* Whether the last action opened a page: a link, back, forward, reload or a
- * jump. Read loosely because Bombadil hands the action over in its trace shape
- * (snake case, empty fields left out), not the one its types declare.
+ * jump. No action at all is the first state, the page the run starts on. Read
+ * loosely because Bombadil hands the action over in its trace shape (snake
+ * case, empty fields left out), not the one its types declare; the tag is
+ * lowercase there, as `getFingerprint` writes it.
*/
const openedPage = extract((state) => {
const action = state.lastAction as unknown;
@@ -86,7 +88,9 @@ const openedPage = extract((state) => {
Click?: { fingerprint: { tag: string; href?: string } };
Custom?: { name: string };
};
- if (Click !== undefined) return Click.fingerprint.tag === 'a' && Click.fingerprint.href !== undefined;
+ if (Click !== undefined) {
+ return Click.fingerprint.tag === 'a' && Click.fingerprint.href !== undefined;
+ }
return Custom !== undefined && (Custom.name === 'goTo' || Custom.name === 'signIn');
});
@@ -97,7 +101,7 @@ const openedPage = extract((state) => {
*/
const accountMenuOpen = extract((state) =>
Array.from(state.document.querySelectorAll('[role="menu"] [role="menuitem"]')).some(
- (item) => (item.textContent ?? '').trim() === SIGN_OUT && item.getClientRects().length > 0
+ (item) => (item.textContent ?? '').trim() === 'Sign out' && item.getClientRects().length > 0
)
);
@@ -107,7 +111,9 @@ const accountMenuOpen = extract((state) =>
* point: Sign out, below the one aimed at.
*/
const accountMenuOpening = extract((state) => {
- const action = state.lastAction as { Click?: { fingerprint: { accessible_name?: string } } } | null;
+ const action = state.lastAction as {
+ Click?: { fingerprint: { accessible_name?: string } };
+ } | null;
return action?.Click?.fingerprint.accessible_name === 'Account menu';
});
@@ -119,21 +125,27 @@ const errorBanners = extract((state) =>
/**
* Every `/gateway/` response since the page last loaded that the server
- * failed. Resource timing keeps 250 entries by default, fewer than a long
- * session on one page makes, so the buffer is raised before it is read.
+ * failed, collected by an observer installed with the first read of each page
+ * load. An observer sees every entry, where the resource timing buffer stops
+ * recording once full; `buffered` hands it what loaded before it was installed.
*/
const serverErrors = extract((state) => {
- const performance = state.window.performance;
- performance.setResourceTimingBufferSize(10_000);
- return performance
- .getEntriesByType('resource')
- .filter(
- (entry): entry is PerformanceResourceTiming =>
- entry instanceof PerformanceResourceTiming &&
- new URL(entry.name).pathname.startsWith('/gateway/') &&
- entry.responseStatus >= 500
- )
- .map((entry) => `${entry.responseStatus} ${new URL(entry.name).pathname}`);
+ const page = state.window as Window & { switchExploreServerErrors?: string[] };
+ if (page.switchExploreServerErrors === undefined) {
+ const failed: string[] = [];
+ page.switchExploreServerErrors = failed;
+ // The extractor runs in the page, so these globals are the page's own.
+ new PerformanceObserver((list) => {
+ for (const entry of list.getEntries()) {
+ if (!(entry instanceof PerformanceResourceTiming) || entry.responseStatus < 500) {
+ continue;
+ }
+ const path = new URL(entry.name).pathname;
+ if (path.startsWith('/gateway/')) failed.push(`${entry.responseStatus} ${path}`);
+ }
+ }).observe({ type: 'resource', buffered: true });
+ }
+ return [...page.switchExploreServerErrors];
});
/** The explorer signs in from the login page it starts on. */
@@ -141,21 +153,25 @@ export const signsIn = eventually(() => signedIn.current).within(30, 'seconds');
/**
* Once signed in, the explorer stays signed in. It cannot sign out (see
- * `SIGN_OUT`), so the login page again means the session was lost.
+ * `SESSION_ENDING`), so the login page again means the session was lost.
*/
export const staysSignedIn = always(
now(() => signedIn.current).implies(always(() => !onLoginPage.current))
);
+const noBanner = () => errorBanners.current.length === 0;
+
/**
* No page opens with an error banner: MUI renders each as an `alert` with error
* severity. A banner after a click is how the gateway refuses what random input
* asked for (removing a member who owns agents, say), and one stays up until
- * dismissed, so only a banner that appears as a page opens counts.
+ * dismissed, so only a banner that appears as a page opens counts: in the
+ * state the page opened in, or the one after, once the explorer has waited for
+ * what the page fetches (see `settling`).
*/
export const noErrorBanner = always(
- now(() => errorBanners.current.length === 0).implies(
- next(now(() => openedPage.current).implies(() => errorBanners.current.length === 0))
+ now(noBanner).implies(
+ next(now(() => openedPage.current).implies(now(noBanner).and(next(noBanner))))
)
);
@@ -193,15 +209,16 @@ const goTo = registerCustomAction('goTo', async (_document, window, route: strin
export const signingIn = actions(() => [signIn()]).when(() => onLoginPage.current);
/**
- * Whether a click is safe: never Sign out, none while the account menu is still
- * opening, and while it is open only on the menu's own entries, so a click
- * cannot land on Sign out by accident.
+ * Whether a click is safe: never on `SESSION_ENDING`, and while the account menu is open
+ * only on the menu's own entries, so a click cannot land on Sign out by
+ * accident.
*/
function safeClick(action: ActionTemplate): boolean {
if (!isAction(action, 'Click')) return true;
- if (accountMenuOpening.current) return false;
const { textContent, accessibleName, role } = action.Click.fingerprint;
- if (textContent === SIGN_OUT || accessibleName === SIGN_OUT) return false;
+ if (SESSION_ENDING.some((label) => textContent === label || accessibleName === label)) {
+ return false;
+ }
return !accountMenuOpen.current || role === 'menuitem' || role === 'radio';
}
@@ -209,6 +226,15 @@ const jumps = actions(() =>
ROUTES.filter((route) => route !== path.current).map((route) => goTo(route))
);
+/**
+ * Whether the explorer should wait a step before acting: a page has just opened
+ * and may still be fetching what it shows (`noErrorBanner` looks at the state
+ * after the wait), or the account menu is still growing into place.
+ */
+const unsettled = () => openedPage.current || accountMenuOpening.current;
+
+export const settling = actions(() => ['Wait']).when(() => signedIn.current && unsettled());
+
/**
* Bombadil's default exploration, without signing out, plus a jump to any page
* in the rail so a dead end (a dialog with nothing to click, a page scrolled
@@ -220,8 +246,7 @@ export const exploring = weighted([
[50, scroll],
[10, navigation],
[5, jumps],
- [100, actions(() => (accountMenuOpening.current ? ['Wait'] : []))],
-]).when(() => signedIn.current);
+]).when(() => signedIn.current && !unsettled());
/**
* Neither signed in nor on the login form: the app is loading its session, or
diff --git a/console/tooling-e2e/src/explore/trace.ts b/console/tooling-e2e/src/explore/trace.ts
index dfd286027..6efd4c997 100644
--- a/console/tooling-e2e/src/explore/trace.ts
+++ b/console/tooling-e2e/src/explore/trace.ts
@@ -98,7 +98,8 @@ export function describeViolation(raw: unknown): string {
return `${describeViolation(violation.And.left)} and ${describeViolation(violation.And.right)}`;
}
if ('Or' in violation) {
- return `${describeViolation(violation.Or.left)} and ${describeViolation(violation.Or.right)}`;
+ const [left, right] = [violation.Or.left, violation.Or.right].map(describeViolation);
+ return `neither ${left} nor ${right}`;
}
if ('Eventually' in violation) {
const { subformula, reason } = violation.Eventually;