Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
ci: Tier 2 wires gbrain MCP into openclaw.json and requires a gbrain tool call; graduation shards require executed tests (#8002, #8003)
garrytan66d557110b96
diff --git a/.github/workflows/e2e.yml b/.github/workflows/e2e.ymlindex 79742d9104..35f9f82bfb 100644--- a/.github/workflows/e2e.yml+++ b/.github/workflows/e2e.yml@@ -529,6 +529,11 @@ jobs:GBRAIN_TEST_RECEIPT_DIR: ${{ runner.temp }}/receiptsGBRAIN_TEST_RECEIPT_LANE: tier1SHARD: ${{ matrix.shard }}/2+ # #8003: the graduation files skip without DATABASE_URL and run-e2e.sh+ # exits 0 on an all-skip run, so a shard with a broken database env+ # fails here instead of passing with nothing executed.+ - name: Require every graduation test to run+ run: bun scripts/require-executed-tests.ts "$RUNNER_TEMP/receipts" tier1- name: Upload receiptsif: always()uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4@@ -798,22 +803,16 @@ jobs:# moments ago) gets up to 4 attempts 20/40/80 s apart. Step cap is a backstop.timeout-minutes: 8run: scripts/ci-install-openclaw.sh+ # #8002: OpenClaw reads MCP servers from mcp.servers in+ # ~/.openclaw/openclaw.json; the script saves gbrain there with+ # `openclaw mcp set`, fails unless `openclaw mcp show` reads it back,+ # and puts scripts/ci-mcp-tap.sh in front of the server so the test step+ # can require a gbrain tool call.- name: Configure OpenClaw MCP- run: |- mkdir -p ~/.openclaw- cat > ~/.openclaw/config.json << 'EOF'- {- "mcpServers": {- "gbrain": {- "command": "bun",- "args": ["run", "src/cli.ts", "serve"],- "env": {- "DATABASE_URL": "${{ env.DATABASE_URL }}"- }- }- }- }- EOF+ run: bash scripts/ci-openclaw-mcp.sh+ env:+ DATABASE_URL: postgresql://postgres:postgres@localhost:5432/gbrain_test+ GBRAIN_TEST_TIER2_MCP_TAP_LOG: ${{ runner.temp }}/gbrain-mcp-tap.jsonl- name: Run Tier 2 skill testsrun: bash scripts/run-with-receipt.sh tier2 skills -- bun test --timeout=60000 test/e2e/skills.test.tsenv:@@ -825,6 +824,8 @@ jobs:# Keyed by decision (GBRA-108: Tier 2 keyed nightly, about $2 a week).GBRAIN_TEST_KEEP_PROVIDER_KEYS: '1'GBRAIN_TEST_RECEIPT_DIR: ${{ runner.temp }}/receipts+ # Each test requires a gbrain tool call in this log (#8002).+ GBRAIN_TEST_TIER2_MCP_TAP_LOG: ${{ runner.temp }}/gbrain-mcp-tap.jsonl# bun test exits 0 when every test skips (an empty secret, a missing# openclaw agent), so a keyless or all-skip run fails here instead.- name: Require every Tier 2 test to rundiff --git a/CHANGELOG.md b/CHANGELOG.mdindex 1d901bb91d..d53421fe9c 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -10,6 +10,19 @@ credits are retained; no result has been reassigned to another provider. Originaidentifiers and attribution are available in the pre-removal Git revision`6040075c6cb95be5881cc2e1b76ef7d71f4e5d29` (retained on 2026-09-23).+## [Unreleased]++**Tier 2's keyed nightly now actually gives the agent gbrain's MCP tools and fails any test in which the agent never called one, and a Tier 1 graduation shard that executed nothing now fails instead of passing.**++Tier 2 wrote its gbrain MCP server into `~/.openclaw/config.json`, which OpenClaw 2026.4.9 never reads, since it reads `mcp.servers` in `~/.openclaw/openclaw.json`. The agent ran with no gbrain tools, and each test checked only that OpenClaw exited 0 and printed something, which it also does when the model key is rejected (#8002). The job now saves the server with `openclaw mcp set` and fails unless `openclaw mcp show` reads it back. The server runs behind `scripts/ci-mcp-tap.sh`, which logs every JSON-RPC request, and each Tier 2 test fails unless the agent made a gbrain tool call while it ran. The `tier1-graduation` shards now run `scripts/require-executed-tests.ts` after their tests, because the graduation files skip without `DATABASE_URL` and `run-e2e.sh` exits 0 on an all-skip run, so a shard with a broken database env used to pass with nothing executed (#8003).++### Itemized changes++- `.github/workflows/e2e.yml`: Tier 2's "Configure OpenClaw MCP" step runs `scripts/ci-openclaw-mcp.sh` and its test step gets `GBRAIN_TEST_TIER2_MCP_TAP_LOG`; each `tier1-graduation` shard runs the executed-tests check on lane `tier1`.+- `scripts/ci-openclaw-mcp.sh`, `scripts/ci-mcp-tap.sh` and `scripts/mcp-tap-calls.ts`: the MCP registration, the request tap and its reader.+- `test/e2e/skills.test.ts`: each test asks for a gbrain MCP tool and requires a gbrain tool call.+- Tests: `test/scripts/tier2-openclaw-mcp.test.ts` and two new cases in `test/scripts/require-executed-tests.test.ts`.+## [0.60.508.0] - 2026-10-30**The E2E full run is shorter and Tier 2 no longer reports green on a run where it executed nothing: Selected E2E uses up to 14 workers, Tier 1's graduation files run in 2 shards of their own, and Tier 2 runs keyed on the nightly schedule with a check that fails when any of its tests skipped.**diff --git a/docs/architecture/key-files/ci-health-and-test-guards.md b/docs/architecture/key-files/ci-health-and-test-guards.mdindex 4513339598..ff56bd845d 100644--- a/docs/architecture/key-files/ci-health-and-test-guards.md+++ b/docs/architecture/key-files/ci-health-and-test-guards.md@@ -16,7 +16,8 @@ Current behavior and load-bearing invariants; history belongs in Git and CHANGEL- `scripts/ci-convergence-scope.sh` — #6712 PR 0: classifies a pull request's changed files (stdin) for the managed-sync convergence gate (`persistence-validation.yml` `convergence`): `run` for persistence, sync commands and sync core, page projections and storage, `bun.lock` and `vendor/postgres/`, the harness (`test/sync-convergence.serial.test.ts`, `test/fixtures/convergence-fault-kill.ts`), this script and its workflows (`persistence-validation.yml`, `convergence-nightly.yml`); `skip` otherwise, and never selected by `package.json` alone, so a release-stamp PR does not run it; an empty list runs. test.yml `changes` calls it for pull requests only (step `convergence`, passed as the `convergence` input). Tests: `test/scripts/ci-pr-scope.test.ts`.- `scripts/ci-manifest-diff.sh` — parsed-JSON classification of manifest diffs for CI scoping: `version-only` (only the top-level `version` differs), `dependency-change` (any dependency field differs), `dependency-scope` (whether a changed-file list touches lockfiles, `patches/` or dependency fields) and `drop-version-only` (filters version-only `package.json` and `openclaw.plugin.json` entries out of a changed-file list, so a release bump does not widen the native scope). Used by test.yml's `changes` job (before `scripts/ci-native-scope.sh`) and its dependency-audit scope step. Tests: `test/scripts/ci-manifest-diff.test.ts`.- `scripts/ci-dependency-audit.sh` — runs `bun audit` for the root and `admin/` lockfiles. It blocks (`BLOCKING=true`) on pushes, schedules and dispatches, on lockfile, `patches/` or dependency-field changes, with the `dependency-audit` label, or when the diff cannot be read; elsewhere an advisory is a `::warning::` naming why the run does not block.-- `scripts/require-executed-tests.ts` — fails a lane whose receipts show zero passed tests, any skipped, todo or failed test, a missing or truncated JUnit report, or no receipt at all (GBRA-135 L6). The keyed Tier 2 nightly (`e2e.yml` `tier2`) runs it after its test step, because `bun test` exits 0 when every key-gated test self-skips. `executedProblems(dir, lane)` reads receipts through `loadReceiptDir` (`scripts/ci-executed-counts.ts`). Test: `test/scripts/require-executed-tests.test.ts` (includes the real keyless `test/e2e/skills.test.ts` run).+- `scripts/require-executed-tests.ts` — fails a lane whose receipts show zero passed tests, any skipped, todo or failed test, a missing or truncated JUnit report, or no receipt at all (GBRA-135 L6). The keyed Tier 2 nightly (`e2e.yml` `tier2`) runs it after its test step, because `bun test` exits 0 when every key-gated test self-skips, and each `tier1-graduation` shard runs it on lane `tier1`, because those files skip without `DATABASE_URL` and `run-e2e.sh` exits 0 (#8003).+- `scripts/ci-openclaw-mcp.sh`, `scripts/ci-mcp-tap.sh`, `scripts/mcp-tap-calls.ts` — Tier 2's gbrain MCP wiring (#8002). The first saves gbrain under `mcp.servers` in `~/.openclaw/openclaw.json` with `openclaw mcp set` and fails unless `openclaw mcp show` reads it back; the server runs behind the tap, which logs every JSON-RPC request to `GBRAIN_TEST_TIER2_MCP_TAP_LOG`; `test/e2e/skills.test.ts` reads that log with `tapToolCalls` and fails a test in which the agent made no gbrain tool call. `executedProblems(dir, lane)` reads receipts through `loadReceiptDir` (`scripts/ci-executed-counts.ts`). Test: `test/scripts/require-executed-tests.test.ts` (includes the real keyless `test/e2e/skills.test.ts` run).- `scripts/ci-install-openclaw.sh` — e2e.yml's Tier 2 `Install OpenClaw` step. Each `npm install -g openclaw@<pin>` attempt is bounded (`timeout 120`); a failure whose npm log carries `code ETARGET` or `E404` is a registry race (a dependency version published but not yet visible for every package of its release, the 2026-10-12 master Tier 2 red on `@aws-sdk/credential-provider-node@^3.972.86`) and is retried up to 4 attempts 20/40/80 s apart; any other failure keeps the 3 attempts 10 s apart. Nothing else is retried, and the last attempt fails the step with npm's own log. Knobs (`OPENCLAW_SPEC`, `OPENCLAW_INSTALL_ATTEMPT_S`, `OPENCLAW_INSTALL_RETRY_SLEEP_S`, `OPENCLAW_INSTALL_RACE_SLEEPS_S`) exist for the test: `test/scripts/ci-install-openclaw.test.ts` installs a one-file package from a loopback registry (`test/helpers/npm-stub-registry.ts`) that answers ETARGET or E404 to the first three packument requests, forever, or publishes for an impossible `os` (`EBADPLATFORM`, the generic path). The lockfile route (a committed `package-lock.json` per pin) was weighed and not taken: two pins live in the workflows (`e2e.yml` 2026.4.9, `native-locks.yml` 2026.9.4), every bump would regenerate a multi-hundred-kilobyte lockfile, and the race the retry waits out is a propagation delay of minutes.- `scripts/e2e-provider-key-notice.sh` — e2e.yml's visible-skip step: when `OPENAI_API_KEY` or `ANTHROPIC_API_KEY` is empty, prints a `::warning::` and step-summary line naming the empty secret and the owner-only fix, so a green keyless run is never read as keyed coverage. Exits 0 always.- `scripts/scale/watchdog.ts` — out-of-process phase watchdog for the scale harness. `scripts/scale/run.ts` re-runs itself as a child; the supervisor times the `import`, `extract`, `vectors` and `budgets` phases from the child's `[scale] phase <name> start` lines and kills the child's process group at 1.5x the phase's gate ceiling (`scripts/scale/gates.ts` `phaseLimitsMs`), names the phase, elapsed time and last progress line, then removes the brain home and scale database. `GBRAIN_SCALE_PHASE_LIMIT_MS_<PHASE>` overrides one limit; `GBRAIN_SCALE_SUPERVISED` is the internal handshake. Tests: `test/scripts/scale-watchdog.test.ts`.diff --git a/docs/ci-event-parity.md b/docs/ci-event-parity.mdindex ab4e013e86..c284c59760 100644--- a/docs/ci-event-parity.md+++ b/docs/ci-event-parity.md@@ -37,7 +37,7 @@ checks ([below](#full-ci-on-pull-requests)), never per-cell matrix names.| `native-locks.yml` | `inputs.scope` (OpenClaw, musl, Windows probes, cross-target cells) | Named exception | Every push to master and the nightly Test schedule (`full`) || `e2e.yml` | Doc-only merge-queue entry selects no E2E file | Named exception | Every push to master (whole corpus) and the nightly E2E schedule || `e2e.yml` | Full-corpus coverage lanes and their `e2e-status` checks | Named exception | Nightly E2E schedule (or a `full_corpus` dispatch) |-| `e2e.yml` | Tier 2 (LLM Skills) runs keyed on the nightly only; other events skip it, and `e2e-status` prints "not run: keys withheld" instead of counting it green | Named exception | Nightly E2E schedule (or a `full_corpus` dispatch), keyed; `scripts/require-executed-tests.ts` fails the job when any test skipped or none ran |+| `e2e.yml` | Tier 2 (LLM Skills) runs keyed on the nightly only; other events skip it, and `e2e-status` prints "not run: keys withheld" instead of counting it green | Named exception | Nightly E2E schedule (or a `full_corpus` dispatch), keyed; `scripts/require-executed-tests.ts` fails the job when any test skipped or none ran, and each test fails unless the agent called a gbrain MCP tool || `heavy-tests.yml` | Label-gated jobs on pull requests | Named exception | Nightly Heavy Tests schedule || `heavy-tests.yml` | `grok-door` (label or input only) | Recorded decision: no scheduled run until the `XAI_API_KEY` secret exists | None yet, by decision || `heavy-tests.yml` | `opencode-door-canary` | Named exception: schedule-only, non-gating canary | Itself |diff --git a/scripts/ci-mcp-tap.sh b/scripts/ci-mcp-tap.shnew file mode 100755index 0000000000..d7bdc24994--- /dev/null+++ b/scripts/ci-mcp-tap.sh@@ -0,0 +1,16 @@+#!/usr/bin/env bash+# Tier 2 (e2e.yml) runs gbrain's stdio MCP server behind this tap (#8002).+# Every JSON-RPC message OpenClaw sends the server is appended to <log>, so+# test/e2e/skills.test.ts can require that the agent made a gbrain tool call+# (scripts/mcp-tap-calls.ts reads it). The server's stdout goes straight back+# to the client, untouched.+#+# Usage: scripts/ci-mcp-tap.sh <log> -- <server command...>+set -uo pipefail+if [ $# -lt 3 ] || [ "$2" != -- ]; then+ echo "usage: scripts/ci-mcp-tap.sh <log> -- <server command...>" >&2+ exit 2+fi+log=$1+shift 2+tee -a "$log" | "$@"diff --git a/scripts/ci-openclaw-mcp.sh b/scripts/ci-openclaw-mcp.shnew file mode 100755index 0000000000..18165676fb--- /dev/null+++ b/scripts/ci-openclaw-mcp.sh@@ -0,0 +1,46 @@+#!/usr/bin/env bash+# Tier 2 (e2e.yml): register gbrain's stdio MCP server with OpenClaw, then+# prove OpenClaw sees it (#8002).+#+# OpenClaw 2026.4.9 reads its MCP servers from `mcp.servers` in+# ~/.openclaw/openclaw.json, and its embedded agent (`openclaw agent --local`)+# loads them from there. The job used to write `mcpServers` into+# ~/.openclaw/config.json, which OpenClaw never reads, so the agent ran with no+# gbrain tools. `openclaw mcp set` writes the file OpenClaw reads, and+# `openclaw mcp show` reads it back.+#+# The server runs behind scripts/ci-mcp-tap.sh, which logs every request to+# $GBRAIN_TEST_TIER2_MCP_TAP_LOG for test/e2e/skills.test.ts.+#+# Environment: DATABASE_URL and GBRAIN_TEST_TIER2_MCP_TAP_LOG (required);+# OPENCLAW, the CLI to call (default openclaw; the test overrides it).+set -euo pipefail+: "${DATABASE_URL:?DATABASE_URL must name the Tier 2 test database}"+: "${GBRAIN_TEST_TIER2_MCP_TAP_LOG:?GBRAIN_TEST_TIER2_MCP_TAP_LOG must name the tap log}"+openclaw=${OPENCLAW:-openclaw}++server=$(GBRAIN_REPO="$PWD" bun -e '+ const e = process.env;+ console.log(JSON.stringify({+ command: "bash",+ args: ["scripts/ci-mcp-tap.sh", e.GBRAIN_TEST_TIER2_MCP_TAP_LOG, "--", "bun", "run", "src/cli.ts", "serve"],+ cwd: e.GBRAIN_REPO,+ env: { DATABASE_URL: e.DATABASE_URL },+ }));+')+: > "$GBRAIN_TEST_TIER2_MCP_TAP_LOG"+"$openclaw" mcp set gbrain "$server"+shown=$("$openclaw" mcp show gbrain --json 2>&1) || true+if ! EXPECTED="$server" SHOWN="$shown" bun -e '+ let b;+ try { b = JSON.parse(process.env.SHOWN); } catch { process.exit(1); }+ const a = JSON.parse(process.env.EXPECTED);+ const same = ["command", "cwd"].every(k => a[k] === b[k])+ && JSON.stringify(a.args) === JSON.stringify(b.args)+ && JSON.stringify(a.env) === JSON.stringify(b.env);+ process.exit(same ? 0 : 1);+'; then+ echo "::error::OpenClaw does not report the gbrain MCP server that was just saved; \`openclaw mcp show gbrain --json\` printed: $shown" >&2+ exit 1+fi+echo "OpenClaw reports the gbrain MCP server: $server"diff --git a/scripts/mcp-tap-calls.ts b/scripts/mcp-tap-calls.tsnew file mode 100644index 0000000000..2356ff0594--- /dev/null+++ b/scripts/mcp-tap-calls.ts@@ -0,0 +1,17 @@+/**+ * Read the JSON-RPC log scripts/ci-mcp-tap.sh writes in front of gbrain's+ * stdio MCP server, and return the name of every tool the client called+ * (`tools/call`), in order (#8002). A line that is not JSON-RPC is ignored.+ */+export function tapToolCalls(log: string): string[] {+ const names: string[] = [];+ for (const line of log.split('\n')) {+ let parsed: unknown;+ try { parsed = JSON.parse(line); } catch { continue; }+ for (const msg of Array.isArray(parsed) ? parsed : [parsed]) {+ const m = msg as { method?: unknown; params?: { name?: unknown } } | null;+ if (m?.method === 'tools/call' && typeof m.params?.name === 'string') names.push(m.params.name);+ }+ }+ return names;+}diff --git a/scripts/structural-suites.tsv b/scripts/structural-suites.tsvindex e4463c1b73..114fa74115 100644--- a/scripts/structural-suites.tsv+++ b/scripts/structural-suites.tsv@@ -340,7 +340,7 @@ test/scripts/postgres-lane-coverage.test.ts check-postgres-lane-coverage.ts 9 retest/scripts/postgres-unit-arms.test.ts postgres-unit-arms list 13 readFileSynctest/scripts/promote-stable.test.ts promote-stable.yml 2 readFileSynctest/scripts/regen-all.test.ts regen:all 5 readFileSync-test/scripts/require-executed-tests.test.ts require-executed-tests 7 readFileSync+test/scripts/require-executed-tests.test.ts require-executed-tests 9 readFileSynctest/scripts/run-verify-parallel.test.ts guard registration ⇒ execution coverage 1 readFileSynctest/scripts/run-verify-parallel.test.ts run-verify-parallel.sh — no-timeout-binary fallback rc capture (regression) 7 readFileSynctest/scripts/serial-files.test.ts scripts/serial-files.tsv (serial lane manifest) 5 readFileSync@@ -351,6 +351,7 @@ test/scripts/stress-plan.test.ts shards 5 readFileSynctest/scripts/test-env-opt-ins-guard.test.ts check-test-env-opt-ins 5 readFileSynctest/scripts/test-shard.slow.test.ts test-shard.sh — LPT balance contract 5 readFileSynctest/scripts/test-shard.slow.test.ts test-shard.sh — exclusion contract 7 readFileSync+test/scripts/tier2-openclaw-mcp.test.ts Tier 2 OpenClaw MCP wiring (#8002) 6 readFileSynctest/scripts/ubicloud-typecheck-state.test.ts (file-level) 3 readFileSynctest/search/graph-signals-wire-integration.test.ts v0.40.4 — source-grep regression guard 1 source-surface-helpertest/serve-behavior-notice-6543.test.ts (file-level) 3 readFileSyncdiff --git a/test/e2e/skills.test.ts b/test/e2e/skills.test.tsindex 3805856a5c..a11c3f3193 100644--- a/test/e2e/skills.test.ts+++ b/test/e2e/skills.test.ts@@ -1,14 +1,19 @@/*** E2E Skill Tests — Tier 2 (requires API keys + openclaw)*- * Tests gbrain skills via OpenClaw agent CLI invocations.- * Asserts on DB state changes, not LLM output text.+ * Tests gbrain skills via OpenClaw agent CLI invocations. Each test requires+ * that the agent called a gbrain MCP tool: OpenClaw exits 0 and prints text+ * even when it never reached gbrain (no MCP server configured, a rejected+ * model key), so exit code and output alone prove nothing (#8002).** Requires:* - DATABASE_URL* - OPENAI_API_KEY* - ANTHROPIC_API_KEY* - openclaw CLI installed with at least one agent configured+ * - GBRAIN_TEST_TIER2_MCP_TAP_LOG: the JSON-RPC log of gbrain's MCP server, which+ * scripts/ci-openclaw-mcp.sh registers with OpenClaw behind+ * scripts/ci-mcp-tap.sh** Skips gracefully if any dependency is missing.* Run: source ~/.zshrc && DATABASE_URL=... bun test test/e2e/skills.test.ts@@ -16,7 +21,9 @@import { describe, test, expect, beforeAll, afterAll } from 'bun:test';import { join } from 'path';+import { existsSync, readFileSync, statSync } from 'fs';import { hasDatabase, setupDB, teardownDB, importFixtures, getEngine } from './helpers.ts';+import { tapToolCalls } from '../../scripts/mcp-tap-calls.ts';// Detect the default openclaw agentfunction detectAgent(): string | null {@@ -42,6 +49,7 @@ function hasTier2Deps(): { ok: boolean; reason?: string; agent?: string } {if (!hasDatabase()) return { ok: false, reason: 'DATABASE_URL not set' };if (!process.env.OPENAI_API_KEY) return { ok: false, reason: 'OPENAI_API_KEY not set' };if (!process.env.ANTHROPIC_API_KEY) return { ok: false, reason: 'ANTHROPIC_API_KEY not set' };+ if (!process.env.GBRAIN_TEST_TIER2_MCP_TAP_LOG) return { ok: false, reason: 'GBRAIN_TEST_TIER2_MCP_TAP_LOG not set (run scripts/ci-openclaw-mcp.sh)' };// Check if openclaw is installedtry {@@ -110,13 +118,24 @@ function runOpenClaw(prompt: string, timeoutMs = 120_000) {};}+/** The gbrain MCP tools the agent called while `run` ran, read from the tap log. */+function gbrainToolCalls<T>(run: () => T): { result: T; calls: string[] } {+ const log = process.env.GBRAIN_TEST_TIER2_MCP_TAP_LOG!;+ const start = existsSync(log) ? statSync(log).size : 0;+ const result = run();+ const calls = existsSync(log) ? tapToolCalls(readFileSync(log).subarray(start).toString('utf8')) : [];+ return { result, calls };+}+// ─────────────────────────────────────────────────────────────────// Ingest Skill// ─────────────────────────────────────────────────────────────────describeT2('E2E Tier 2: Ingest Skill', () => {- // Note: the agent uses its own configured DB, not the test DB.- // We verify the agent responds, not DB state changes.+ beforeAll(async () => {+ await setupDB();+ }, 30_000);+ afterAll(teardownDB);test('ingest a meeting transcript creates person pages and links', async () => {const transcript = `@@ -130,16 +149,14 @@ Decision: Hire VP Sales by end of Q2.Action: Sarah to draft VP Sales job description by April 7.`.trim();- const { text, exitCode, durationMs } = runOpenClaw(- `Ingest this meeting transcript into gbrain. Create or update pages for each person mentioned. Add timeline entries for today's date. Here is the transcript:\n\n${transcript}`,+ const { result: { text, exitCode, durationMs }, calls } = gbrainToolCalls(() => runOpenClaw(+ `Using the gbrain MCP tools, ingest this meeting transcript into gbrain. Create or update pages for each person mentioned. Add timeline entries for today's date. Here is the transcript:\n\n${transcript}`,180_000,- );+ ));- console.log(` Ingest skill completed in ${durationMs}ms`);+ console.log(` Ingest skill completed in ${durationMs}ms; gbrain tool calls: ${calls.join(', ') || 'none'}`);- // The agent runs against its own configured gbrain DB, not our test DB.- // We can't assert on test DB state. Instead, verify the agent responded- // with content indicating it processed the transcript.+ expect(calls.length, `the agent never called a gbrain MCP tool; it said: ${text.slice(0, 500)}`).toBeGreaterThan(0);expect(text.length).toBeGreaterThan(0);expect(exitCode).toBe(0);}, 240_000);@@ -157,14 +174,14 @@ describeT2('E2E Tier 2: Query Skill', () => {afterAll(teardownDB);test('query skill returns results for known topic', async () => {- const { text, exitCode, durationMs } = runOpenClaw(- 'Search gbrain for "NovaMind" and tell me what you found.',+ const { result: { text, exitCode, durationMs }, calls } = gbrainToolCalls(() => runOpenClaw(+ 'Use the gbrain MCP search tool to search gbrain for "NovaMind" and tell me what you found.',180_000,- );+ ));- console.log(` Query skill completed in ${durationMs}ms`);+ console.log(` Query skill completed in ${durationMs}ms; gbrain tool calls: ${calls.join(', ') || 'none'}`);- // The agent should have responded with something+ expect(calls.length, `the agent never called a gbrain MCP tool; it said: ${text.slice(0, 500)}`).toBeGreaterThan(0);expect(text.length).toBeGreaterThan(0);expect(exitCode).toBe(0);}, 240_000);@@ -182,13 +199,14 @@ describeT2('E2E Tier 2: Health Skill', () => {afterAll(teardownDB);test('health skill reports brain status', async () => {- const { text, exitCode, durationMs } = runOpenClaw(- 'Run gbrain doctor --json and tell me the results.',+ const { result: { text, exitCode, durationMs }, calls } = gbrainToolCalls(() => runOpenClaw(+ 'Use the gbrain MCP get_health tool to check the brain\'s health and tell me the results.',180_000,- );+ ));- console.log(` Health skill completed in ${durationMs}ms`);+ console.log(` Health skill completed in ${durationMs}ms; gbrain tool calls: ${calls.join(', ') || 'none'}`);+ expect(calls.length, `the agent never called a gbrain MCP tool; it said: ${text.slice(0, 500)}`).toBeGreaterThan(0);expect(text.length).toBeGreaterThan(0);expect(exitCode).toBe(0);}, 240_000);diff --git a/test/scripts/require-executed-tests.test.ts b/test/scripts/require-executed-tests.test.tsindex 844d31e1ee..d68fa4c749 100644--- a/test/scripts/require-executed-tests.test.ts+++ b/test/scripts/require-executed-tests.test.ts@@ -89,4 +89,29 @@ describe('require-executed-tests', () => {expect(names.indexOf('Require every Tier 2 test to run')).toBe(names.indexOf('Run Tier 2 skill tests') + 1);expect(tier2.steps.find(s => s.name === 'Provider-key coverage notice')!.env!.GBRAIN_TEST_KEEP_PROVIDER_KEYS).toBe('1');});++ test('a graduation shard with no database env skips everything and exits 0, and the guard fails it (#8003)', () => {+ const receipts = mkdtempSync(join(tmpdir(), 'gbrain-require-executed-grad-'));+ try {+ const env: Record<string, string | undefined> = { ...process.env, GBRAIN_NO_SNAPSHOT: '1', GBRAIN_TEST_RECEIPT_DIR: receipts, GBRAIN_TEST_RECEIPT_LANE: 'tier1', SHARD: '1/1' };+ delete env.DATABASE_URL;+ delete env.GBRAIN_PGBOUNCER_URL;+ const run = spawnSync('bash', ['scripts/run-e2e.sh', 'test/e2e/graduation-cli-history.test.ts'], { cwd: REPO, encoding: 'utf8', env });+ expect(run.status, run.stdout + run.stderr).toBe(0);+ const g = guard(receipts, 'tier1');+ expect(g.status).toBe(1);+ expect(g.stderr).toContain('tier1 ran 0 tests');+ expect(g.stderr).toContain('skip: test/e2e/graduation-cli-history.test.ts > graduation: 1k-page history round trip');+ } finally { rmSync(receipts, { recursive: true, force: true }); }+ });++ test('e2e.yml gates each graduation shard on the guard (#8003)', () => {+ type Step = { name?: string; run?: string; if?: string };+ const grad = (load(readFileSync(join(REPO, '.github/workflows/e2e.yml'), 'utf8')) as { jobs: Record<string, { steps: Step[] }> }).jobs['tier1-graduation']!;+ const names = grad.steps.map(s => s.name);+ const check = grad.steps.find(s => s.name === 'Require every graduation test to run')!;+ expect(check.run).toBe('bun scripts/require-executed-tests.ts "$RUNNER_TEMP/receipts" tier1');+ expect(check.if).toBeUndefined();+ expect(names.indexOf('Require every graduation test to run')).toBe(names.indexOf('Run engine graduation suites (direct Postgres and PgBouncer)') + 1);+ });});diff --git a/test/scripts/tier2-openclaw-mcp.test.ts b/test/scripts/tier2-openclaw-mcp.test.tsnew file mode 100644index 0000000000..a2437411fa--- /dev/null+++ b/test/scripts/tier2-openclaw-mcp.test.ts@@ -0,0 +1,121 @@+/**+ * #8002: Tier 2 (e2e.yml) must wire gbrain's MCP server into the file+ * OpenClaw 2026.4.9 reads, and every Tier 2 test must prove the agent called+ * a gbrain tool. The job used to write `mcpServers` into+ * ~/.openclaw/config.json, which OpenClaw never reads (it reads+ * `mcp.servers` in ~/.openclaw/openclaw.json), so the agent ran with no gbrain+ * tools while every test checked only exit code and non-empty output.+ */+import { describe, expect, test } from 'bun:test';+import { chmodSync, existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';+import { tmpdir } from 'node:os';+import { join } from 'node:path';+import { spawnSync } from 'node:child_process';+import { load } from 'js-yaml';+import { tapToolCalls } from '../../scripts/mcp-tap-calls.ts';++const REPO = join(import.meta.dir, '../..');+type Step = { name?: string; run?: string; env?: Record<string, string> };+const tier2 = (load(readFileSync(join(REPO, '.github/workflows/e2e.yml'), 'utf8')) as { jobs: Record<string, { steps: Step[] }> }).jobs.tier2!;++/** A stand-in for the openclaw CLI: `mcp set` stores the JSON, `mcp show` prints it (or drops it when DROP=1). */+function fakeOpenclaw(dir: string): string {+ const bin = join(dir, 'openclaw');+ writeFileSync(bin, `#!/usr/bin/env bash+store="${dir}/saved.json"+case "$1 $2" in+ "mcp set") printf '%s' "$4" > "$store" ;;+ "mcp show") if [ "\${DROP:-}" = 1 ] || [ ! -f "$store" ]; then echo 'No MCP server named "gbrain"'; exit 1; fi; cat "$store" ;;+ *) exit 64 ;;+esac+`);+ chmodSync(bin, 0o755);+ return bin;+}++function configure(env: Record<string, string | undefined>) {+ return spawnSync('bash', ['scripts/ci-openclaw-mcp.sh'], { cwd: REPO, encoding: 'utf8', env: { ...process.env, ...env } });+}++describe('Tier 2 OpenClaw MCP wiring (#8002)', () => {+ test('e2e.yml registers gbrain through scripts/ci-openclaw-mcp.sh, never through config.json', () => {+ const cfg = tier2.steps.find(s => s.name === 'Configure OpenClaw MCP')!;+ expect(cfg.run).toBe('bash scripts/ci-openclaw-mcp.sh');+ expect(cfg.env!.DATABASE_URL).toBe('postgresql://postgres:postgres@localhost:5432/gbrain_test');+ for (const step of tier2.steps) {+ expect(step.run ?? '').not.toContain('.openclaw/config.json');+ expect(step.run ?? '').not.toContain('mcpServers');+ }+ const run = tier2.steps.find(s => s.name === 'Run Tier 2 skill tests')!;+ expect(cfg.env!.GBRAIN_TEST_TIER2_MCP_TAP_LOG).toBe('${{ runner.temp }}/gbrain-mcp-tap.jsonl');+ expect(run.env!.GBRAIN_TEST_TIER2_MCP_TAP_LOG).toBe(cfg.env!.GBRAIN_TEST_TIER2_MCP_TAP_LOG);+ const names = tier2.steps.map(s => s.name);+ expect(names.indexOf('Configure OpenClaw MCP')).toBeLessThan(names.indexOf('Run Tier 2 skill tests'));+ });++ test('the script saves gbrain behind the tap with openclaw mcp set and checks it reads back', () => {+ const dir = mkdtempSync(join(tmpdir(), 'gbrain-oc-mcp-'));+ try {+ const tap = join(dir, 'tap.jsonl');+ writeFileSync(tap, 'stale\n');+ const r = configure({ OPENCLAW: fakeOpenclaw(dir), DATABASE_URL: 'postgresql://u@localhost:5432/db', GBRAIN_TEST_TIER2_MCP_TAP_LOG: tap });+ expect(r.status, r.stdout + r.stderr).toBe(0);+ expect(JSON.parse(readFileSync(join(dir, 'saved.json'), 'utf8'))).toEqual({+ command: 'bash',+ args: ['scripts/ci-mcp-tap.sh', tap, '--', 'bun', 'run', 'src/cli.ts', 'serve'],+ cwd: REPO,+ env: { DATABASE_URL: 'postgresql://u@localhost:5432/db' },+ });+ expect(readFileSync(tap, 'utf8')).toBe('');+ } finally { rmSync(dir, { recursive: true, force: true }); }+ });++ test('the script fails when OpenClaw does not report the saved server, or an input is missing', () => {+ const dir = mkdtempSync(join(tmpdir(), 'gbrain-oc-mcp-'));+ try {+ const base = { OPENCLAW: fakeOpenclaw(dir), DATABASE_URL: 'postgresql://u@localhost/db', GBRAIN_TEST_TIER2_MCP_TAP_LOG: join(dir, 't') };+ const dropped = configure({ ...base, DROP: '1' });+ expect(dropped.status).not.toBe(0);+ expect(dropped.stderr).toContain('OpenClaw does not report the gbrain MCP server');+ expect(configure({ ...base, DATABASE_URL: undefined }).status).not.toBe(0);+ expect(configure({ ...base, GBRAIN_TEST_TIER2_MCP_TAP_LOG: undefined }).status).not.toBe(0);+ } finally { rmSync(dir, { recursive: true, force: true }); }+ });++ test('the tap passes the stream through unchanged and logs what the client sent', () => {+ const dir = mkdtempSync(join(tmpdir(), 'gbrain-mcp-tap-'));+ try {+ const log = join(dir, 'tap.jsonl');+ const input = [+ '{"jsonrpc":"2.0","id":0,"method":"initialize","params":{}}',+ '{"jsonrpc":"2.0","method":"notifications/initialized"}',+ '{"jsonrpc":"2.0","id":1,"method":"tools/list"}',+ '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"search","arguments":{"query":"x"}}}',+ ].join('\n') + '\n';+ const r = spawnSync('bash', ['scripts/ci-mcp-tap.sh', log, '--', 'cat'], { cwd: REPO, input, encoding: 'utf8' });+ expect(r.status, r.stderr).toBe(0);+ expect(r.stdout).toBe(input);+ expect(readFileSync(log, 'utf8')).toBe(input);+ expect(tapToolCalls(readFileSync(log, 'utf8'))).toEqual(['search']);+ expect(spawnSync('bash', ['scripts/ci-mcp-tap.sh', log, 'cat'], { cwd: REPO }).status).toBe(2);+ } finally { rmSync(dir, { recursive: true, force: true }); }+ });++ test('tapToolCalls counts only tools/call requests, batches included', () => {+ const handshakeOnly = '{"jsonrpc":"2.0","id":0,"method":"initialize"}\n{"jsonrpc":"2.0","id":1,"method":"tools/list"}\n';+ expect(tapToolCalls(handshakeOnly)).toEqual([]);+ expect(tapToolCalls('')).toEqual([]);+ expect(tapToolCalls('not json\n{"method":"tools/call","params":{}}\n')).toEqual([]);+ expect(tapToolCalls('[{"method":"tools/call","params":{"name":"put_page"}},{"method":"ping"}]\n{"method":"tools/call","params":{"name":"get_health"}}')).toEqual(['put_page', 'get_health']);+ });++ test('every Tier 2 test requires a gbrain tool call, and the file needs the tap log to run', () => {+ const src = readFileSync(join(REPO, 'test/e2e/skills.test.ts'), 'utf8');+ const runs = src.match(/runOpenClaw\(\n/g) ?? [];+ expect(runs.length).toBe(3);+ expect(src.match(/gbrainToolCalls\(\(\) => runOpenClaw\(/g)).toHaveLength(3);+ expect(src.match(/expect\(calls\.length, /g)).toHaveLength(3);+ expect(src).toContain("if (!process.env.GBRAIN_TEST_TIER2_MCP_TAP_LOG) return { ok: false");+ expect(existsSync(join(REPO, 'scripts/ci-mcp-tap.sh'))).toBe(true);+ });+});