Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

ci: Tier 2 wires gbrain MCP into openclaw.json and requires a gbrain tool call; graduation shards require executed tests (#8002, #8003)

garrytan66d557110b96
diff --git a/.github/workflows/e2e.yml b/.github/workflows/e2e.yml
index 79742d9104..35f9f82bfb 100644
--- a/.github/workflows/e2e.yml
+++ b/.github/workflows/e2e.yml
@@ -529,6 +529,11 @@ jobs:
GBRAIN_TEST_RECEIPT_DIR: ${{ runner.temp }}/receipts
GBRAIN_TEST_RECEIPT_LANE: tier1
SHARD: ${{ matrix.shard }}/2
+ # #8003: the graduation files skip without DATABASE_URL and run-e2e.sh
+ # exits 0 on an all-skip run, so a shard with a broken database env
+ # fails here instead of passing with nothing executed.
+ - name: Require every graduation test to run
+ run: bun scripts/require-executed-tests.ts "$RUNNER_TEMP/receipts" tier1
- name: Upload receipts
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
@@ -798,22 +803,16 @@ jobs:
# moments ago) gets up to 4 attempts 20/40/80 s apart. Step cap is a backstop.
timeout-minutes: 8
run: scripts/ci-install-openclaw.sh
+ # #8002: OpenClaw reads MCP servers from mcp.servers in
+ # ~/.openclaw/openclaw.json; the script saves gbrain there with
+ # `openclaw mcp set`, fails unless `openclaw mcp show` reads it back,
+ # and puts scripts/ci-mcp-tap.sh in front of the server so the test step
+ # can require a gbrain tool call.
- name: Configure OpenClaw MCP
- run: |
- mkdir -p ~/.openclaw
- cat > ~/.openclaw/config.json << 'EOF'
- {
- "mcpServers": {
- "gbrain": {
- "command": "bun",
- "args": ["run", "src/cli.ts", "serve"],
- "env": {
- "DATABASE_URL": "${{ env.DATABASE_URL }}"
- }
- }
- }
- }
- EOF
+ run: bash scripts/ci-openclaw-mcp.sh
+ env:
+ DATABASE_URL: postgresql://postgres:postgres@localhost:5432/gbrain_test
+ GBRAIN_TEST_TIER2_MCP_TAP_LOG: ${{ runner.temp }}/gbrain-mcp-tap.jsonl
- name: Run Tier 2 skill tests
run: bash scripts/run-with-receipt.sh tier2 skills -- bun test --timeout=60000 test/e2e/skills.test.ts
env:
@@ -825,6 +824,8 @@ jobs:
# Keyed by decision (GBRA-108: Tier 2 keyed nightly, about $2 a week).
GBRAIN_TEST_KEEP_PROVIDER_KEYS: '1'
GBRAIN_TEST_RECEIPT_DIR: ${{ runner.temp }}/receipts
+ # Each test requires a gbrain tool call in this log (#8002).
+ GBRAIN_TEST_TIER2_MCP_TAP_LOG: ${{ runner.temp }}/gbrain-mcp-tap.jsonl
# bun test exits 0 when every test skips (an empty secret, a missing
# openclaw agent), so a keyless or all-skip run fails here instead.
- name: Require every Tier 2 test to run
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1d901bb91d..d53421fe9c 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,6 +10,19 @@ credits are retained; no result has been reassigned to another provider. Origina
identifiers and attribution are available in the pre-removal Git revision
`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).
+## [Unreleased]
+
+**Tier 2's keyed nightly now actually gives the agent gbrain's MCP tools and fails any test in which the agent never called one, and a Tier 1 graduation shard that executed nothing now fails instead of passing.**
+
+Tier 2 wrote its gbrain MCP server into `~/.openclaw/config.json`, which OpenClaw 2026.4.9 never reads, since it reads `mcp.servers` in `~/.openclaw/openclaw.json`. The agent ran with no gbrain tools, and each test checked only that OpenClaw exited 0 and printed something, which it also does when the model key is rejected (#8002). The job now saves the server with `openclaw mcp set` and fails unless `openclaw mcp show` reads it back. The server runs behind `scripts/ci-mcp-tap.sh`, which logs every JSON-RPC request, and each Tier 2 test fails unless the agent made a gbrain tool call while it ran. The `tier1-graduation` shards now run `scripts/require-executed-tests.ts` after their tests, because the graduation files skip without `DATABASE_URL` and `run-e2e.sh` exits 0 on an all-skip run, so a shard with a broken database env used to pass with nothing executed (#8003).
+
+### Itemized changes
+
+- `.github/workflows/e2e.yml`: Tier 2's "Configure OpenClaw MCP" step runs `scripts/ci-openclaw-mcp.sh` and its test step gets `GBRAIN_TEST_TIER2_MCP_TAP_LOG`; each `tier1-graduation` shard runs the executed-tests check on lane `tier1`.
+- `scripts/ci-openclaw-mcp.sh`, `scripts/ci-mcp-tap.sh` and `scripts/mcp-tap-calls.ts`: the MCP registration, the request tap and its reader.
+- `test/e2e/skills.test.ts`: each test asks for a gbrain MCP tool and requires a gbrain tool call.
+- Tests: `test/scripts/tier2-openclaw-mcp.test.ts` and two new cases in `test/scripts/require-executed-tests.test.ts`.
+
## [0.60.508.0] - 2026-10-30
**The E2E full run is shorter and Tier 2 no longer reports green on a run where it executed nothing: Selected E2E uses up to 14 workers, Tier 1's graduation files run in 2 shards of their own, and Tier 2 runs keyed on the nightly schedule with a check that fails when any of its tests skipped.**
diff --git a/docs/architecture/key-files/ci-health-and-test-guards.md b/docs/architecture/key-files/ci-health-and-test-guards.md
index 4513339598..ff56bd845d 100644
--- a/docs/architecture/key-files/ci-health-and-test-guards.md
+++ b/docs/architecture/key-files/ci-health-and-test-guards.md
@@ -16,7 +16,8 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL
- `scripts/ci-convergence-scope.sh` — #6712 PR 0: classifies a pull request's changed files (stdin) for the managed-sync convergence gate (`persistence-validation.yml` `convergence`): `run` for persistence, sync commands and sync core, page projections and storage, `bun.lock` and `vendor/postgres/`, the harness (`test/sync-convergence.serial.test.ts`, `test/fixtures/convergence-fault-kill.ts`), this script and its workflows (`persistence-validation.yml`, `convergence-nightly.yml`); `skip` otherwise, and never selected by `package.json` alone, so a release-stamp PR does not run it; an empty list runs. test.yml `changes` calls it for pull requests only (step `convergence`, passed as the `convergence` input). Tests: `test/scripts/ci-pr-scope.test.ts`.
- `scripts/ci-manifest-diff.sh` — parsed-JSON classification of manifest diffs for CI scoping: `version-only` (only the top-level `version` differs), `dependency-change` (any dependency field differs), `dependency-scope` (whether a changed-file list touches lockfiles, `patches/` or dependency fields) and `drop-version-only` (filters version-only `package.json` and `openclaw.plugin.json` entries out of a changed-file list, so a release bump does not widen the native scope). Used by test.yml's `changes` job (before `scripts/ci-native-scope.sh`) and its dependency-audit scope step. Tests: `test/scripts/ci-manifest-diff.test.ts`.
- `scripts/ci-dependency-audit.sh` — runs `bun audit` for the root and `admin/` lockfiles. It blocks (`BLOCKING=true`) on pushes, schedules and dispatches, on lockfile, `patches/` or dependency-field changes, with the `dependency-audit` label, or when the diff cannot be read; elsewhere an advisory is a `::warning::` naming why the run does not block.
-- `scripts/require-executed-tests.ts` — fails a lane whose receipts show zero passed tests, any skipped, todo or failed test, a missing or truncated JUnit report, or no receipt at all (GBRA-135 L6). The keyed Tier 2 nightly (`e2e.yml` `tier2`) runs it after its test step, because `bun test` exits 0 when every key-gated test self-skips. `executedProblems(dir, lane)` reads receipts through `loadReceiptDir` (`scripts/ci-executed-counts.ts`). Test: `test/scripts/require-executed-tests.test.ts` (includes the real keyless `test/e2e/skills.test.ts` run).
+- `scripts/require-executed-tests.ts` — fails a lane whose receipts show zero passed tests, any skipped, todo or failed test, a missing or truncated JUnit report, or no receipt at all (GBRA-135 L6). The keyed Tier 2 nightly (`e2e.yml` `tier2`) runs it after its test step, because `bun test` exits 0 when every key-gated test self-skips, and each `tier1-graduation` shard runs it on lane `tier1`, because those files skip without `DATABASE_URL` and `run-e2e.sh` exits 0 (#8003).
+- `scripts/ci-openclaw-mcp.sh`, `scripts/ci-mcp-tap.sh`, `scripts/mcp-tap-calls.ts` — Tier 2's gbrain MCP wiring (#8002). The first saves gbrain under `mcp.servers` in `~/.openclaw/openclaw.json` with `openclaw mcp set` and fails unless `openclaw mcp show` reads it back; the server runs behind the tap, which logs every JSON-RPC request to `GBRAIN_TEST_TIER2_MCP_TAP_LOG`; `test/e2e/skills.test.ts` reads that log with `tapToolCalls` and fails a test in which the agent made no gbrain tool call. `executedProblems(dir, lane)` reads receipts through `loadReceiptDir` (`scripts/ci-executed-counts.ts`). Test: `test/scripts/require-executed-tests.test.ts` (includes the real keyless `test/e2e/skills.test.ts` run).
- `scripts/ci-install-openclaw.sh` — e2e.yml's Tier 2 `Install OpenClaw` step. Each `npm install -g openclaw@<pin>` attempt is bounded (`timeout 120`); a failure whose npm log carries `code ETARGET` or `E404` is a registry race (a dependency version published but not yet visible for every package of its release, the 2026-10-12 master Tier 2 red on `@aws-sdk/credential-provider-node@^3.972.86`) and is retried up to 4 attempts 20/40/80 s apart; any other failure keeps the 3 attempts 10 s apart. Nothing else is retried, and the last attempt fails the step with npm's own log. Knobs (`OPENCLAW_SPEC`, `OPENCLAW_INSTALL_ATTEMPT_S`, `OPENCLAW_INSTALL_RETRY_SLEEP_S`, `OPENCLAW_INSTALL_RACE_SLEEPS_S`) exist for the test: `test/scripts/ci-install-openclaw.test.ts` installs a one-file package from a loopback registry (`test/helpers/npm-stub-registry.ts`) that answers ETARGET or E404 to the first three packument requests, forever, or publishes for an impossible `os` (`EBADPLATFORM`, the generic path). The lockfile route (a committed `package-lock.json` per pin) was weighed and not taken: two pins live in the workflows (`e2e.yml` 2026.4.9, `native-locks.yml` 2026.9.4), every bump would regenerate a multi-hundred-kilobyte lockfile, and the race the retry waits out is a propagation delay of minutes.
- `scripts/e2e-provider-key-notice.sh` — e2e.yml's visible-skip step: when `OPENAI_API_KEY` or `ANTHROPIC_API_KEY` is empty, prints a `::warning::` and step-summary line naming the empty secret and the owner-only fix, so a green keyless run is never read as keyed coverage. Exits 0 always.
- `scripts/scale/watchdog.ts` — out-of-process phase watchdog for the scale harness. `scripts/scale/run.ts` re-runs itself as a child; the supervisor times the `import`, `extract`, `vectors` and `budgets` phases from the child's `[scale] phase <name> start` lines and kills the child's process group at 1.5x the phase's gate ceiling (`scripts/scale/gates.ts` `phaseLimitsMs`), names the phase, elapsed time and last progress line, then removes the brain home and scale database. `GBRAIN_SCALE_PHASE_LIMIT_MS_<PHASE>` overrides one limit; `GBRAIN_SCALE_SUPERVISED` is the internal handshake. Tests: `test/scripts/scale-watchdog.test.ts`.
diff --git a/docs/ci-event-parity.md b/docs/ci-event-parity.md
index ab4e013e86..c284c59760 100644
--- a/docs/ci-event-parity.md
+++ b/docs/ci-event-parity.md
@@ -37,7 +37,7 @@ checks ([below](#full-ci-on-pull-requests)), never per-cell matrix names.
| `native-locks.yml` | `inputs.scope` (OpenClaw, musl, Windows probes, cross-target cells) | Named exception | Every push to master and the nightly Test schedule (`full`) |
| `e2e.yml` | Doc-only merge-queue entry selects no E2E file | Named exception | Every push to master (whole corpus) and the nightly E2E schedule |
| `e2e.yml` | Full-corpus coverage lanes and their `e2e-status` checks | Named exception | Nightly E2E schedule (or a `full_corpus` dispatch) |
-| `e2e.yml` | Tier 2 (LLM Skills) runs keyed on the nightly only; other events skip it, and `e2e-status` prints "not run: keys withheld" instead of counting it green | Named exception | Nightly E2E schedule (or a `full_corpus` dispatch), keyed; `scripts/require-executed-tests.ts` fails the job when any test skipped or none ran |
+| `e2e.yml` | Tier 2 (LLM Skills) runs keyed on the nightly only; other events skip it, and `e2e-status` prints "not run: keys withheld" instead of counting it green | Named exception | Nightly E2E schedule (or a `full_corpus` dispatch), keyed; `scripts/require-executed-tests.ts` fails the job when any test skipped or none ran, and each test fails unless the agent called a gbrain MCP tool |
| `heavy-tests.yml` | Label-gated jobs on pull requests | Named exception | Nightly Heavy Tests schedule |
| `heavy-tests.yml` | `grok-door` (label or input only) | Recorded decision: no scheduled run until the `XAI_API_KEY` secret exists | None yet, by decision |
| `heavy-tests.yml` | `opencode-door-canary` | Named exception: schedule-only, non-gating canary | Itself |
diff --git a/scripts/ci-mcp-tap.sh b/scripts/ci-mcp-tap.sh
new file mode 100755
index 0000000000..d7bdc24994
--- /dev/null
+++ b/scripts/ci-mcp-tap.sh
@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+# Tier 2 (e2e.yml) runs gbrain's stdio MCP server behind this tap (#8002).
+# Every JSON-RPC message OpenClaw sends the server is appended to <log>, so
+# test/e2e/skills.test.ts can require that the agent made a gbrain tool call
+# (scripts/mcp-tap-calls.ts reads it). The server's stdout goes straight back
+# to the client, untouched.
+#
+# Usage: scripts/ci-mcp-tap.sh <log> -- <server command...>
+set -uo pipefail
+if [ $# -lt 3 ] || [ "$2" != -- ]; then
+ echo "usage: scripts/ci-mcp-tap.sh <log> -- <server command...>" >&2
+ exit 2
+fi
+log=$1
+shift 2
+tee -a "$log" | "$@"
diff --git a/scripts/ci-openclaw-mcp.sh b/scripts/ci-openclaw-mcp.sh
new file mode 100755
index 0000000000..18165676fb
--- /dev/null
+++ b/scripts/ci-openclaw-mcp.sh
@@ -0,0 +1,46 @@
+#!/usr/bin/env bash
+# Tier 2 (e2e.yml): register gbrain's stdio MCP server with OpenClaw, then
+# prove OpenClaw sees it (#8002).
+#
+# OpenClaw 2026.4.9 reads its MCP servers from `mcp.servers` in
+# ~/.openclaw/openclaw.json, and its embedded agent (`openclaw agent --local`)
+# loads them from there. The job used to write `mcpServers` into
+# ~/.openclaw/config.json, which OpenClaw never reads, so the agent ran with no
+# gbrain tools. `openclaw mcp set` writes the file OpenClaw reads, and
+# `openclaw mcp show` reads it back.
+#
+# The server runs behind scripts/ci-mcp-tap.sh, which logs every request to
+# $GBRAIN_TEST_TIER2_MCP_TAP_LOG for test/e2e/skills.test.ts.
+#
+# Environment: DATABASE_URL and GBRAIN_TEST_TIER2_MCP_TAP_LOG (required);
+# OPENCLAW, the CLI to call (default openclaw; the test overrides it).
+set -euo pipefail
+: "${DATABASE_URL:?DATABASE_URL must name the Tier 2 test database}"
+: "${GBRAIN_TEST_TIER2_MCP_TAP_LOG:?GBRAIN_TEST_TIER2_MCP_TAP_LOG must name the tap log}"
+openclaw=${OPENCLAW:-openclaw}
+
+server=$(GBRAIN_REPO="$PWD" bun -e '
+ const e = process.env;
+ console.log(JSON.stringify({
+ command: "bash",
+ args: ["scripts/ci-mcp-tap.sh", e.GBRAIN_TEST_TIER2_MCP_TAP_LOG, "--", "bun", "run", "src/cli.ts", "serve"],
+ cwd: e.GBRAIN_REPO,
+ env: { DATABASE_URL: e.DATABASE_URL },
+ }));
+')
+: > "$GBRAIN_TEST_TIER2_MCP_TAP_LOG"
+"$openclaw" mcp set gbrain "$server"
+shown=$("$openclaw" mcp show gbrain --json 2>&1) || true
+if ! EXPECTED="$server" SHOWN="$shown" bun -e '
+ let b;
+ try { b = JSON.parse(process.env.SHOWN); } catch { process.exit(1); }
+ const a = JSON.parse(process.env.EXPECTED);
+ const same = ["command", "cwd"].every(k => a[k] === b[k])
+ && JSON.stringify(a.args) === JSON.stringify(b.args)
+ && JSON.stringify(a.env) === JSON.stringify(b.env);
+ process.exit(same ? 0 : 1);
+'; then
+ echo "::error::OpenClaw does not report the gbrain MCP server that was just saved; \`openclaw mcp show gbrain --json\` printed: $shown" >&2
+ exit 1
+fi
+echo "OpenClaw reports the gbrain MCP server: $server"
diff --git a/scripts/mcp-tap-calls.ts b/scripts/mcp-tap-calls.ts
new file mode 100644
index 0000000000..2356ff0594
--- /dev/null
+++ b/scripts/mcp-tap-calls.ts
@@ -0,0 +1,17 @@
+/**
+ * Read the JSON-RPC log scripts/ci-mcp-tap.sh writes in front of gbrain's
+ * stdio MCP server, and return the name of every tool the client called
+ * (`tools/call`), in order (#8002). A line that is not JSON-RPC is ignored.
+ */
+export function tapToolCalls(log: string): string[] {
+ const names: string[] = [];
+ for (const line of log.split('\n')) {
+ let parsed: unknown;
+ try { parsed = JSON.parse(line); } catch { continue; }
+ for (const msg of Array.isArray(parsed) ? parsed : [parsed]) {
+ const m = msg as { method?: unknown; params?: { name?: unknown } } | null;
+ if (m?.method === 'tools/call' && typeof m.params?.name === 'string') names.push(m.params.name);
+ }
+ }
+ return names;
+}
diff --git a/scripts/structural-suites.tsv b/scripts/structural-suites.tsv
index e4463c1b73..114fa74115 100644
--- a/scripts/structural-suites.tsv
+++ b/scripts/structural-suites.tsv
@@ -340,7 +340,7 @@ test/scripts/postgres-lane-coverage.test.ts check-postgres-lane-coverage.ts 9 re
test/scripts/postgres-unit-arms.test.ts postgres-unit-arms list 13 readFileSync
test/scripts/promote-stable.test.ts promote-stable.yml 2 readFileSync
test/scripts/regen-all.test.ts regen:all 5 readFileSync
-test/scripts/require-executed-tests.test.ts require-executed-tests 7 readFileSync
+test/scripts/require-executed-tests.test.ts require-executed-tests 9 readFileSync
test/scripts/run-verify-parallel.test.ts guard registration ⇒ execution coverage 1 readFileSync
test/scripts/run-verify-parallel.test.ts run-verify-parallel.sh — no-timeout-binary fallback rc capture (regression) 7 readFileSync
test/scripts/serial-files.test.ts scripts/serial-files.tsv (serial lane manifest) 5 readFileSync
@@ -351,6 +351,7 @@ test/scripts/stress-plan.test.ts shards 5 readFileSync
test/scripts/test-env-opt-ins-guard.test.ts check-test-env-opt-ins 5 readFileSync
test/scripts/test-shard.slow.test.ts test-shard.sh — LPT balance contract 5 readFileSync
test/scripts/test-shard.slow.test.ts test-shard.sh — exclusion contract 7 readFileSync
+test/scripts/tier2-openclaw-mcp.test.ts Tier 2 OpenClaw MCP wiring (#8002) 6 readFileSync
test/scripts/ubicloud-typecheck-state.test.ts (file-level) 3 readFileSync
test/search/graph-signals-wire-integration.test.ts v0.40.4 — source-grep regression guard 1 source-surface-helper
test/serve-behavior-notice-6543.test.ts (file-level) 3 readFileSync
diff --git a/test/e2e/skills.test.ts b/test/e2e/skills.test.ts
index 3805856a5c..a11c3f3193 100644
--- a/test/e2e/skills.test.ts
+++ b/test/e2e/skills.test.ts
@@ -1,14 +1,19 @@
/**
* E2E Skill Tests — Tier 2 (requires API keys + openclaw)
*
- * Tests gbrain skills via OpenClaw agent CLI invocations.
- * Asserts on DB state changes, not LLM output text.
+ * Tests gbrain skills via OpenClaw agent CLI invocations. Each test requires
+ * that the agent called a gbrain MCP tool: OpenClaw exits 0 and prints text
+ * even when it never reached gbrain (no MCP server configured, a rejected
+ * model key), so exit code and output alone prove nothing (#8002).
*
* Requires:
* - DATABASE_URL
* - OPENAI_API_KEY
* - ANTHROPIC_API_KEY
* - openclaw CLI installed with at least one agent configured
+ * - GBRAIN_TEST_TIER2_MCP_TAP_LOG: the JSON-RPC log of gbrain's MCP server, which
+ * scripts/ci-openclaw-mcp.sh registers with OpenClaw behind
+ * scripts/ci-mcp-tap.sh
*
* Skips gracefully if any dependency is missing.
* Run: source ~/.zshrc && DATABASE_URL=... bun test test/e2e/skills.test.ts
@@ -16,7 +21,9 @@
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { join } from 'path';
+import { existsSync, readFileSync, statSync } from 'fs';
import { hasDatabase, setupDB, teardownDB, importFixtures, getEngine } from './helpers.ts';
+import { tapToolCalls } from '../../scripts/mcp-tap-calls.ts';
// Detect the default openclaw agent
function detectAgent(): string | null {
@@ -42,6 +49,7 @@ function hasTier2Deps(): { ok: boolean; reason?: string; agent?: string } {
if (!hasDatabase()) return { ok: false, reason: 'DATABASE_URL not set' };
if (!process.env.OPENAI_API_KEY) return { ok: false, reason: 'OPENAI_API_KEY not set' };
if (!process.env.ANTHROPIC_API_KEY) return { ok: false, reason: 'ANTHROPIC_API_KEY not set' };
+ if (!process.env.GBRAIN_TEST_TIER2_MCP_TAP_LOG) return { ok: false, reason: 'GBRAIN_TEST_TIER2_MCP_TAP_LOG not set (run scripts/ci-openclaw-mcp.sh)' };
// Check if openclaw is installed
try {
@@ -110,13 +118,24 @@ function runOpenClaw(prompt: string, timeoutMs = 120_000) {
};
}
+/** The gbrain MCP tools the agent called while `run` ran, read from the tap log. */
+function gbrainToolCalls<T>(run: () => T): { result: T; calls: string[] } {
+ const log = process.env.GBRAIN_TEST_TIER2_MCP_TAP_LOG!;
+ const start = existsSync(log) ? statSync(log).size : 0;
+ const result = run();
+ const calls = existsSync(log) ? tapToolCalls(readFileSync(log).subarray(start).toString('utf8')) : [];
+ return { result, calls };
+}
+
// ─────────────────────────────────────────────────────────────────
// Ingest Skill
// ─────────────────────────────────────────────────────────────────
describeT2('E2E Tier 2: Ingest Skill', () => {
- // Note: the agent uses its own configured DB, not the test DB.
- // We verify the agent responds, not DB state changes.
+ beforeAll(async () => {
+ await setupDB();
+ }, 30_000);
+ afterAll(teardownDB);
test('ingest a meeting transcript creates person pages and links', async () => {
const transcript = `
@@ -130,16 +149,14 @@ Decision: Hire VP Sales by end of Q2.
Action: Sarah to draft VP Sales job description by April 7.
`.trim();
- const { text, exitCode, durationMs } = runOpenClaw(
- `Ingest this meeting transcript into gbrain. Create or update pages for each person mentioned. Add timeline entries for today's date. Here is the transcript:\n\n${transcript}`,
+ const { result: { text, exitCode, durationMs }, calls } = gbrainToolCalls(() => runOpenClaw(
+ `Using the gbrain MCP tools, ingest this meeting transcript into gbrain. Create or update pages for each person mentioned. Add timeline entries for today's date. Here is the transcript:\n\n${transcript}`,
180_000,
- );
+ ));
- console.log(` Ingest skill completed in ${durationMs}ms`);
+ console.log(` Ingest skill completed in ${durationMs}ms; gbrain tool calls: ${calls.join(', ') || 'none'}`);
- // The agent runs against its own configured gbrain DB, not our test DB.
- // We can't assert on test DB state. Instead, verify the agent responded
- // with content indicating it processed the transcript.
+ expect(calls.length, `the agent never called a gbrain MCP tool; it said: ${text.slice(0, 500)}`).toBeGreaterThan(0);
expect(text.length).toBeGreaterThan(0);
expect(exitCode).toBe(0);
}, 240_000);
@@ -157,14 +174,14 @@ describeT2('E2E Tier 2: Query Skill', () => {
afterAll(teardownDB);
test('query skill returns results for known topic', async () => {
- const { text, exitCode, durationMs } = runOpenClaw(
- 'Search gbrain for "NovaMind" and tell me what you found.',
+ const { result: { text, exitCode, durationMs }, calls } = gbrainToolCalls(() => runOpenClaw(
+ 'Use the gbrain MCP search tool to search gbrain for "NovaMind" and tell me what you found.',
180_000,
- );
+ ));
- console.log(` Query skill completed in ${durationMs}ms`);
+ console.log(` Query skill completed in ${durationMs}ms; gbrain tool calls: ${calls.join(', ') || 'none'}`);
- // The agent should have responded with something
+ expect(calls.length, `the agent never called a gbrain MCP tool; it said: ${text.slice(0, 500)}`).toBeGreaterThan(0);
expect(text.length).toBeGreaterThan(0);
expect(exitCode).toBe(0);
}, 240_000);
@@ -182,13 +199,14 @@ describeT2('E2E Tier 2: Health Skill', () => {
afterAll(teardownDB);
test('health skill reports brain status', async () => {
- const { text, exitCode, durationMs } = runOpenClaw(
- 'Run gbrain doctor --json and tell me the results.',
+ const { result: { text, exitCode, durationMs }, calls } = gbrainToolCalls(() => runOpenClaw(
+ 'Use the gbrain MCP get_health tool to check the brain\'s health and tell me the results.',
180_000,
- );
+ ));
- console.log(` Health skill completed in ${durationMs}ms`);
+ console.log(` Health skill completed in ${durationMs}ms; gbrain tool calls: ${calls.join(', ') || 'none'}`);
+ expect(calls.length, `the agent never called a gbrain MCP tool; it said: ${text.slice(0, 500)}`).toBeGreaterThan(0);
expect(text.length).toBeGreaterThan(0);
expect(exitCode).toBe(0);
}, 240_000);
diff --git a/test/scripts/require-executed-tests.test.ts b/test/scripts/require-executed-tests.test.ts
index 844d31e1ee..d68fa4c749 100644
--- a/test/scripts/require-executed-tests.test.ts
+++ b/test/scripts/require-executed-tests.test.ts
@@ -89,4 +89,29 @@ describe('require-executed-tests', () => {
expect(names.indexOf('Require every Tier 2 test to run')).toBe(names.indexOf('Run Tier 2 skill tests') + 1);
expect(tier2.steps.find(s => s.name === 'Provider-key coverage notice')!.env!.GBRAIN_TEST_KEEP_PROVIDER_KEYS).toBe('1');
});
+
+ test('a graduation shard with no database env skips everything and exits 0, and the guard fails it (#8003)', () => {
+ const receipts = mkdtempSync(join(tmpdir(), 'gbrain-require-executed-grad-'));
+ try {
+ const env: Record<string, string | undefined> = { ...process.env, GBRAIN_NO_SNAPSHOT: '1', GBRAIN_TEST_RECEIPT_DIR: receipts, GBRAIN_TEST_RECEIPT_LANE: 'tier1', SHARD: '1/1' };
+ delete env.DATABASE_URL;
+ delete env.GBRAIN_PGBOUNCER_URL;
+ const run = spawnSync('bash', ['scripts/run-e2e.sh', 'test/e2e/graduation-cli-history.test.ts'], { cwd: REPO, encoding: 'utf8', env });
+ expect(run.status, run.stdout + run.stderr).toBe(0);
+ const g = guard(receipts, 'tier1');
+ expect(g.status).toBe(1);
+ expect(g.stderr).toContain('tier1 ran 0 tests');
+ expect(g.stderr).toContain('skip: test/e2e/graduation-cli-history.test.ts > graduation: 1k-page history round trip');
+ } finally { rmSync(receipts, { recursive: true, force: true }); }
+ });
+
+ test('e2e.yml gates each graduation shard on the guard (#8003)', () => {
+ type Step = { name?: string; run?: string; if?: string };
+ const grad = (load(readFileSync(join(REPO, '.github/workflows/e2e.yml'), 'utf8')) as { jobs: Record<string, { steps: Step[] }> }).jobs['tier1-graduation']!;
+ const names = grad.steps.map(s => s.name);
+ const check = grad.steps.find(s => s.name === 'Require every graduation test to run')!;
+ expect(check.run).toBe('bun scripts/require-executed-tests.ts "$RUNNER_TEMP/receipts" tier1');
+ expect(check.if).toBeUndefined();
+ expect(names.indexOf('Require every graduation test to run')).toBe(names.indexOf('Run engine graduation suites (direct Postgres and PgBouncer)') + 1);
+ });
});
diff --git a/test/scripts/tier2-openclaw-mcp.test.ts b/test/scripts/tier2-openclaw-mcp.test.ts
new file mode 100644
index 0000000000..a2437411fa
--- /dev/null
+++ b/test/scripts/tier2-openclaw-mcp.test.ts
@@ -0,0 +1,121 @@
+/**
+ * #8002: Tier 2 (e2e.yml) must wire gbrain's MCP server into the file
+ * OpenClaw 2026.4.9 reads, and every Tier 2 test must prove the agent called
+ * a gbrain tool. The job used to write `mcpServers` into
+ * ~/.openclaw/config.json, which OpenClaw never reads (it reads
+ * `mcp.servers` in ~/.openclaw/openclaw.json), so the agent ran with no gbrain
+ * tools while every test checked only exit code and non-empty output.
+ */
+import { describe, expect, test } from 'bun:test';
+import { chmodSync, existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { spawnSync } from 'node:child_process';
+import { load } from 'js-yaml';
+import { tapToolCalls } from '../../scripts/mcp-tap-calls.ts';
+
+const REPO = join(import.meta.dir, '../..');
+type Step = { name?: string; run?: string; env?: Record<string, string> };
+const tier2 = (load(readFileSync(join(REPO, '.github/workflows/e2e.yml'), 'utf8')) as { jobs: Record<string, { steps: Step[] }> }).jobs.tier2!;
+
+/** A stand-in for the openclaw CLI: `mcp set` stores the JSON, `mcp show` prints it (or drops it when DROP=1). */
+function fakeOpenclaw(dir: string): string {
+ const bin = join(dir, 'openclaw');
+ writeFileSync(bin, `#!/usr/bin/env bash
+store="${dir}/saved.json"
+case "$1 $2" in
+ "mcp set") printf '%s' "$4" > "$store" ;;
+ "mcp show") if [ "\${DROP:-}" = 1 ] || [ ! -f "$store" ]; then echo 'No MCP server named "gbrain"'; exit 1; fi; cat "$store" ;;
+ *) exit 64 ;;
+esac
+`);
+ chmodSync(bin, 0o755);
+ return bin;
+}
+
+function configure(env: Record<string, string | undefined>) {
+ return spawnSync('bash', ['scripts/ci-openclaw-mcp.sh'], { cwd: REPO, encoding: 'utf8', env: { ...process.env, ...env } });
+}
+
+describe('Tier 2 OpenClaw MCP wiring (#8002)', () => {
+ test('e2e.yml registers gbrain through scripts/ci-openclaw-mcp.sh, never through config.json', () => {
+ const cfg = tier2.steps.find(s => s.name === 'Configure OpenClaw MCP')!;
+ expect(cfg.run).toBe('bash scripts/ci-openclaw-mcp.sh');
+ expect(cfg.env!.DATABASE_URL).toBe('postgresql://postgres:postgres@localhost:5432/gbrain_test');
+ for (const step of tier2.steps) {
+ expect(step.run ?? '').not.toContain('.openclaw/config.json');
+ expect(step.run ?? '').not.toContain('mcpServers');
+ }
+ const run = tier2.steps.find(s => s.name === 'Run Tier 2 skill tests')!;
+ expect(cfg.env!.GBRAIN_TEST_TIER2_MCP_TAP_LOG).toBe('${{ runner.temp }}/gbrain-mcp-tap.jsonl');
+ expect(run.env!.GBRAIN_TEST_TIER2_MCP_TAP_LOG).toBe(cfg.env!.GBRAIN_TEST_TIER2_MCP_TAP_LOG);
+ const names = tier2.steps.map(s => s.name);
+ expect(names.indexOf('Configure OpenClaw MCP')).toBeLessThan(names.indexOf('Run Tier 2 skill tests'));
+ });
+
+ test('the script saves gbrain behind the tap with openclaw mcp set and checks it reads back', () => {
+ const dir = mkdtempSync(join(tmpdir(), 'gbrain-oc-mcp-'));
+ try {
+ const tap = join(dir, 'tap.jsonl');
+ writeFileSync(tap, 'stale\n');
+ const r = configure({ OPENCLAW: fakeOpenclaw(dir), DATABASE_URL: 'postgresql://u@localhost:5432/db', GBRAIN_TEST_TIER2_MCP_TAP_LOG: tap });
+ expect(r.status, r.stdout + r.stderr).toBe(0);
+ expect(JSON.parse(readFileSync(join(dir, 'saved.json'), 'utf8'))).toEqual({
+ command: 'bash',
+ args: ['scripts/ci-mcp-tap.sh', tap, '--', 'bun', 'run', 'src/cli.ts', 'serve'],
+ cwd: REPO,
+ env: { DATABASE_URL: 'postgresql://u@localhost:5432/db' },
+ });
+ expect(readFileSync(tap, 'utf8')).toBe('');
+ } finally { rmSync(dir, { recursive: true, force: true }); }
+ });
+
+ test('the script fails when OpenClaw does not report the saved server, or an input is missing', () => {
+ const dir = mkdtempSync(join(tmpdir(), 'gbrain-oc-mcp-'));
+ try {
+ const base = { OPENCLAW: fakeOpenclaw(dir), DATABASE_URL: 'postgresql://u@localhost/db', GBRAIN_TEST_TIER2_MCP_TAP_LOG: join(dir, 't') };
+ const dropped = configure({ ...base, DROP: '1' });
+ expect(dropped.status).not.toBe(0);
+ expect(dropped.stderr).toContain('OpenClaw does not report the gbrain MCP server');
+ expect(configure({ ...base, DATABASE_URL: undefined }).status).not.toBe(0);
+ expect(configure({ ...base, GBRAIN_TEST_TIER2_MCP_TAP_LOG: undefined }).status).not.toBe(0);
+ } finally { rmSync(dir, { recursive: true, force: true }); }
+ });
+
+ test('the tap passes the stream through unchanged and logs what the client sent', () => {
+ const dir = mkdtempSync(join(tmpdir(), 'gbrain-mcp-tap-'));
+ try {
+ const log = join(dir, 'tap.jsonl');
+ const input = [
+ '{"jsonrpc":"2.0","id":0,"method":"initialize","params":{}}',
+ '{"jsonrpc":"2.0","method":"notifications/initialized"}',
+ '{"jsonrpc":"2.0","id":1,"method":"tools/list"}',
+ '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"search","arguments":{"query":"x"}}}',
+ ].join('\n') + '\n';
+ const r = spawnSync('bash', ['scripts/ci-mcp-tap.sh', log, '--', 'cat'], { cwd: REPO, input, encoding: 'utf8' });
+ expect(r.status, r.stderr).toBe(0);
+ expect(r.stdout).toBe(input);
+ expect(readFileSync(log, 'utf8')).toBe(input);
+ expect(tapToolCalls(readFileSync(log, 'utf8'))).toEqual(['search']);
+ expect(spawnSync('bash', ['scripts/ci-mcp-tap.sh', log, 'cat'], { cwd: REPO }).status).toBe(2);
+ } finally { rmSync(dir, { recursive: true, force: true }); }
+ });
+
+ test('tapToolCalls counts only tools/call requests, batches included', () => {
+ const handshakeOnly = '{"jsonrpc":"2.0","id":0,"method":"initialize"}\n{"jsonrpc":"2.0","id":1,"method":"tools/list"}\n';
+ expect(tapToolCalls(handshakeOnly)).toEqual([]);
+ expect(tapToolCalls('')).toEqual([]);
+ expect(tapToolCalls('not json\n{"method":"tools/call","params":{}}\n')).toEqual([]);
+ expect(tapToolCalls('[{"method":"tools/call","params":{"name":"put_page"}},{"method":"ping"}]\n{"method":"tools/call","params":{"name":"get_health"}}')).toEqual(['put_page', 'get_health']);
+ });
+
+ test('every Tier 2 test requires a gbrain tool call, and the file needs the tap log to run', () => {
+ const src = readFileSync(join(REPO, 'test/e2e/skills.test.ts'), 'utf8');
+ const runs = src.match(/runOpenClaw\(\n/g) ?? [];
+ expect(runs.length).toBe(3);
+ expect(src.match(/gbrainToolCalls\(\(\) => runOpenClaw\(/g)).toHaveLength(3);
+ expect(src.match(/expect\(calls\.length, /g)).toHaveLength(3);
+ expect(src).toContain("if (!process.env.GBRAIN_TEST_TIER2_MCP_TAP_LOG) return { ok: false");
+ expect(existsSync(join(REPO, 'scripts/ci-mcp-tap.sh'))).toBe(true);
+ });
+});