Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript
25.7 KB
Raw Blame

Key files: Mcp

Subsystem index. Read only the entries relevant to your change. Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG.

  • src/mcp/dispatch.ts — shared tool-call dispatch consumed by both stdio (server.ts) and HTTP transports. Exports dispatchToolCall(engine, name, params, opts) (a read op (mutating === false) runs inside withRequestConfigSnapshot, a write op under withoutRequestConfigSnapshot; src/core/config-snapshot.ts), buildOperationContext(engine, params, opts), and re-exports normalizeOptionalParams/validateParams from src/mcp/validate-params.ts (the validation module — call order normalize → validate → findUnknownParams is load-bearing). Single source of truth for (ctx, params) handler arg order and the OperationContext shape. Defaults remote: true (untrusted); local CLI callers pass remote: false. Deny layers, in order: opts.allowedOps surface enforcement (hidden op → unknownToolEnvelope, byte-identical to a nonexistent op; did-you-mean candidates drawn ONLY from the caller's visible surface minus localOnly minus gated names), unknown op, the localOnly transport backstop (op.localOnly && transport !== 'stdio' → same envelope; stdio IS the local surface), param validation, strict/warn unknown-arg handling (reject mode returns invalid_params with suggestions; warn mode attaches _meta.warnings + a model-visible notice block), then enforceBoundClientOpAllowList (the bound-client fence) inside the handler try. _meta assembly per docs/protocol/MCP_META_CHANNELS.md: handler-emitted keys via ctx.emitResponseMeta (retrieval, warnings) attach first and independently of the metaHook (brain_hot_memory, built by src/core/facts/meta-hook.ts; for a read op on a live engine it runs beside the handler instead of after it (GBRA-156), and reads its withdrawal watermark and trust generation in one round trip — its 30s cache is keyed by ENGINE IDENTITY + source + visibility tier + session + allow-list hash + eligibility + the sessionOnly scope (a session with no facts of its own falls back to the source's newest 24h facts unless the caller pas... (line truncated to 2000 chars)

  • src/core/egress-budget.ts — the per-connection result cap: resultByteCap (Infinity for the trusted local CLI; else mcp.max_result_bytes_stdio 4 MiB / mcp.max_result_bytes_http 1 MiB, DB plane then file plane, clamped to 256 KiB–9 MiB), encodedFrameBytes (the complete JSON-RPC frame plus a framing reserve), and the dispatcher backstop pair frameOverCap / resultTooLargeError.

  • src/mcp/instructions.ts — canonical MCP initialize-time agent operating contract, compile-time source text (no runtime filesystem dependency) shared by the stdio SDK server, the OAuth HTTP SDK server and the legacy raw HTTP initialize path. buildMcpInstructions({writeback?, tools?}) GENERATES the contract from the caller's callable set (tools.callable, the tools/list predicate): a clause naming a tool (search/query/get_page, skills catalog, put_page, whoami, join/sync skills, put/delete skill, the memory loop's context_pack/volunteer_context/remember/forget) appears only when that tool is listed, so every tool named is in that surface's tools/list (test/mcp-initialize-instructions.test.ts). When callable tools are left out of the listed set (hiddenCallable) and request_tools is callable, the hidden-tool sentence leads the contract, inside the 2,048 characters capped harnesses read (D1). It always carries the error protocol (follow fix.next, then fix.verify) and the notice prefix, then the optional status-only line (F4), the opt-in ambient-writeback section (src/core/facts/writeback-instructions.ts) and the readiness tail (top two missing/degraded gaps among embeddings, chat_llm, migrations, worker — never a by-choice state). With no tools and no writeback it equals GBRAIN_MCP_INSTRUCTIONS (the every-tool rendering bootstrap writes to GBRAIN-INSTRUCTIONS.md). resolveMcpInstructions(config, env?, opts) appends the operator-set deployment identity LAST under Deployment identity: (GBRAIN_MCP_INSTRUCTIONS env when non-blank, else file-plane mcp.instructions). installInstructionsResolver(server, compute) swaps the SDK Server's initialize handler so the stdio and OAuth servers compute the contract (callable set + readiness tail via src/mcp/initialize-context.ts) when the client initializes; the constructor value stays the fallback on a throw. The legacy bearer transport builds it inline per initialize. Transport tests compare... (line truncated to 2000 chars)

  • src/mcp/publish-gates.ts — publish-gate resolution for the honest tools/list. readPublishGate(engine, config, key): dual-plane (DB > file > false), never throws — a FAILED read resolves false (hide-on-doubt matches the default-off consent posture). disabledOpsForPublishGates(engine, config): the op-name set tools/list subtracts; one getConfig read per distinct gate key per call, deliberately NOT memoized so gbrain config set mcp.publish_skills true takes effect on the next list without a restart. Call-time gates inside the handlers stay as the fail-closed backstop (their denials carry detail: 'config_key=<key>' — the machine-readable denial grammar). Pinned by test/publish-gates.test.ts. PUBLISH_GATE_KEYS (every operation's gate key) is the set gbrain config get resolves DB-first, matching this resolver (test/config-get-plane.test.ts).

  • src/mcp/rate-limit.ts — Bounded-LRU token-bucket limiter. buildDefaultLimiters() returns the two-bucket pipeline: pre-auth IP (30/60s, fires BEFORE the DB lookup so brute-force load against access_tokens is capped) + post-auth token-id (60/60s). Tracks lastTouchedMs separately from lastRefillMs so an exhausted key can't be reset by hammering past the TTL. LRU cap bounds memory under attacker-controlled key growth. refund(key) returns one consumed token (capped at the limit) — the request_tools persist path refunds when a race-lost 0-row UPDATE (concurrent operator pin) means no write actually happened; dry-run previews never draw a token at all (the limiter meters actual writes only).

  • src/mcp/remote-bridge.ts — the thin-client stdio bridge (startRemoteStdioBridge): bare gbrain serve on a thin client, engine-free. tools/list is the host's list for the stored OAuth client (listRemoteToolDefs), narrowed only by the local --surface/--advertise through the same StdioSurfaceState and stdioToolListing as local stdio; a narrowed listing drops host tools this build does not know (logged once). tools/call goes through forwardRemoteToolCall with client name gbrain-stdio-bridge and _meta unchanged; localOnly/cliOnly and out-of-surface names are refused as unknown_tool before forwarding. request_tools {surface} widens only this session (never forwarded); a relayed request_tools {tools} answer reveals in-surface names (then tools/list_changed) and gets a session note. An audience call (_meta.on_behalf_of) needs a host at AUDIENCE_MIN_HOST_VERSION (hostAtLeast on serverInfo) and a read answer that reports _meta.access.audience, else host_upgrade_required. Lifecycle via installStdioShutdownTriggers; the EOF drain bound is the call timeout, an expired drain with a write in flight logs it and exits 1, and MCP_STDIO=1 keeps the process alive. Seams: transport, lifecycle, list, forward. Pinned by test/remote-stdio-bridge.serial.test.ts (real serve --http host) and test/remote-bridge-errors.test.ts.

  • src/mcp/remote-bridge-errors.ts — the bridge's v1 envelopes: a per-RemoteMcpError reason fix table (exhaustive), writeIdentity (not_sent resends; an accepted journaled mutation is write_outcome_unknown with its request_id and a get_write_request fix; submit_* points at list_jobs), bridgeCallFailure/bridgeListFailure (list: JSON-RPC InternalError whose data is the envelope plus stage, side_effects: none and bounded recovery), writeBridgeTrace (0600 under bridge-traces/), bridgeRefusal. No fix ever adds a scope; every string passes stripUrlSecrets.

  • src/mcp/remote-bridge-args.ts — parseBridgeServeArgs (bare, --surface, --advertise, --source-guard bridge; a repeat takes the narrowest; a bad value is invalid_params) and serveThinClientRefusal (requires_local_engine; the fix keeps the refused flags, unsafe values masked; --access gets the narrow read-only client recipe). Engine- and SDK-free, loaded by src/commands/thin-client-routing.ts. Pinned by test/remote-bridge-args.test.ts and the thin-client goldens.

  • src/mcp/result-rows.ts — the transport side of the search/query row shape. resolveResultRowsMode(engine, config): dual-plane mcp.result_rows (DB > file > 'lean'; a failed DB read falls to the file plane). resultRowsForRequest(header, hostMode): the X-Gbrain-Client header naming gbrain-remote-cli (sent by src/core/mcp-client.ts) selects full rows, otherwise the host mode. The OAuth HTTP transport resolves both per request, the legacy bearer transport per tools/call, and stdio once at start (file plane only when the engine is degraded). The header is unverified and selects a row shape only; it must never gate anything security-relevant. Pinned by test/mcp-result-rows-skew.test.ts.

  • src/mcp/resolve-ipc-binding.ts — shared resolve-IPC listener wiring for BOTH serve transports. bindResolveIpcForServe(engine, defaultSource) binds the retrieval-reflex resolve, turn_context, and context_pack kinds (on Postgres the socket is keyed by hash12(database_url) plus hash12(source) of the serve's bound source under ~/.gbrain/run, and the legacy URL-only socket is also bound when free; hooks pick theirs with hookResolveSocketForConfig — own source first, legacy fallback; the secret stays URL-keyed and the persistence socket brain-keyed; the bound-source posture rejects requests naming any other source; pinned by test/resolve-ipc-source-keyed.test.ts) plus the serve-delegated sync (sync_start/sync_status/sync_abort → src/core/serve-sync-runner.ts) and maintenance-sweep (sweep_start/sweep_status → src/core/serve-sweep-runner.ts) kinds — each delegation family in its OWN try/catch, so a runner failure logs [serve-sync]/[serve-sweep] handlers unavailable and the core kinds still start; both families share the GBRAIN_SERVE_SYNC_IPC=0 kill switch. Engine-uniform: Postgres brains listen too (the hook lane is engine-free by design, so IPC through a serve is its only DB path there). Best-effort by contract — a bind failure never blocks the serve, and a socket already owned by a live or unresponsive listener (another serve) is left to it (that serve is the IPC provider, #4896). Each socket the serve could not bind (its source socket, the legacy socket, also when only the persistence listener came up) is retried on its own unref'd timer (RESOLVE_IPC_REBIND_INTERVAL_MS = 30 s plus up to 5 s jitter, rebindDelayMs injectable) through the same startResolveIpcServer, so a surviving serve takes over once the owner probes dead and logs [resolve-ipc] took over <path>; bind attempts are serialized per serve, close() clears the timers and closes a bind that lands after it, and persistence is never re-set-up (#6647, pinned by `te... (line truncated to 2000 chars)

  • src/mcp/server.ts — MCP stdio server. initialize and tools/list are built from OPERATION_MANIFEST (src/core/operation-manifest.generated.ts, every op minus its handler) so the handler graph behind src/core/operations.ts stays unloaded until it is needed; byte-identity with the eager registry across surfaces, strict params, read-only access, publish gates and the advertised surface is pinned by test/mcp-tool-list-snapshot.test.ts. Engine-dependent boot (bootEngineDependents: source scope, persistence consumer, resolve-IPC bind, startup sweep; #7599: a read-only stdio serve on Postgres (readOnlyServeSkipsResidentDuties) binds IPC with no delegation kinds (lean), starts no consumer, runs no session-cursor GC or startup sweep and writes READ_ONLY_SERVE_NOTE to stderr; PGLite keeps all of it) and the dispatch/hot-memory imports wait for the handshake (deferredBoot, boot phase initialize_wait): they start when the first tools/list is answered, BOOT_LIST_GRACE_MS after initialize with no tools/list in flight, or BOOT_WITHOUT_INITIALIZE_MS after connect for a client that never initializes. Every tools/call and skill resource read awaits that boot, and a boot failure rejects them with the boot error while startMcpServer still rejects with it. Status-only mode (F4, src/mcp/status-mode.ts): over a status engine it lists only gbrain_status, answers other tools with one serve_status_only block, and every tool call re-probes and may recover in place (then tools/list_changed). gbrain://capabilities carries readiness[], the probed worker status (stdioCapabilityReadiness), and surface/surface_source. One StdioSurfaceState (createStdioSurfaceState in surface.ts) per process is the allow-set tools/list, dispatch (allowedOps, stdioSurface), skill resources and gbrain_status read; request_tools {surface} widens it for the session (stderr surface_widened audit line, then tools/list_changed), and mcp.allow_session_widen is ... (line truncated to 2000 chars)

  • src/mcp/source-preflight.ts — assertStdioSourceBindable(engine, env?): stdio-lane boot preflight called first thing in startMcpServer, before any transport attaches. A well-formed GBRAIN_SOURCE that names no ACTIVE source row (SELECT id FROM sources WHERE id = $1 AND archived = false, the same predicate as assertSourceExists in src/core/source-resolver.ts) throws a message naming the value and the fix (gbrain sources list), so serve exits 1 instead of binding every call to a phantom scope where reads return [] and writes die on the sources FK while health stays green. An inherited __all__ emits an engine-free stderr warning with a source-binding recovery action, without widening its fail-closed scope. Unset env and malformed values are left to the resolver; an engine error fail-opens (guards config, not connectivity); a degraded engine (isEngineDegraded) is never touched so the boot does not spend the proxy's reconnect attempt. HTTP is out of scope (tokens carry their own source grant). Pinned by test/mcp-stdio-source-preflight.test.ts and real PostgreSQL/PGLite processes in test/e2e/mcp-search-transport-matrix.test.ts.

  • src/mcp/stdio-lifecycle.ts — engine-free stdio plumbing shared by local serve and the bridge: mcpStdioTransport/MCP_STDIO_MAX_MESSAGE_BYTES, resolveEofDrainMs, isPidAlive/readLiveParentPid/probeWatchdogAvailable, and installStdioShutdownTriggers (signals, the bounded stdin-EOF drain with onDrainExpired, the MCP_STDIO=1 and TTY skips, the parent watchdog). Pinned by test/serve-stdio-lifecycle.test.ts and test/serve-stdin-eof-drain.test.ts.

  • src/core/mcp-registration.ts — REGISTRATION_SURFACE (full since S0, wave 0: callable and advertised; README, docs/mcp, INSTALL_FOR_AGENTS.md and the plugin manifests name the same value, which the S0 grep test in test/mcp-registration.test.ts keeps so), stdioServeArgv(bin, surface = REGISTRATION_SURFACE) (null → bare serve) and registeredSurface(text) (an existing entry's pinned surface, null for bare, undefined when no serve command is visible): every stdio registration gbrain writes builds from them (readiness harness_wiring, init's quickstart, bootstrap hooks for Claude Code / Codex / opencode, the plugin generator, which also refuses root plugin manifests pinned elsewhere). The OpenClaw manifest stays a bare serve. Also the HTTP+bearer wiring helpers connect.ts re-exports. Test: test/mcp-registration.test.ts.

  • src/core/serve-log.ts — serve's own log hygiene (#7246), installed by src/cli/commands/serve.ts before runServe. For each of stderr, and stdout under --http only (stdio serve's stdout is the JSON-RPC channel), whose descriptor is a regular file, it routes console and stream writes through a ServeLogFile: LineDeduper writes the first occurrence of a line at once and counts repeats inside the window (a leading timestamp ignored), then writes one [gbrain serve-log] repeated N more time(s) in the last Ws: <line> line when the window ends or the process exits; rotation is copy-then-truncate on the inherited append-mode descriptor (<log>.N-1 → <log>.N, <log> → <log>.1, truncate), each write placed at the file's current end, and only when the descriptor's path is known (/proc/self/fd or the serveLogPath/serveErrPath inode) and still names the same inode. TTYs, pipes and journald are untouched. GBRAIN_SERVE_LOG_MAX_BYTES (10 MiB; 0 off), GBRAIN_SERVE_LOG_MAX_FILES (5), GBRAIN_SERVE_LOG_DEDUPE_SECONDS (600; 0 off). Tests: test/serve-log-7246.test.ts.

  • src/core/ops/request-tools.ts — request_tools: catalog (no args), descriptors ({tools}; D1: an empty list or one with no caller-visible name returns the catalog plus did_you_mean drawn only from visibleOpsForCaller, never echoing the requested names, so an invisible existing op and a nonexistent one answer byte-identically), and {surface}: per OAuth client on HTTP (ceiling, operator lock, rate limit, audit), and on the owner's stdio pipe widenStdioSession widens ctx.stdioSurface for the session (refused with the persistent route when mcp.allow_session_widen is off; clamped by GBRAIN_MCP_FORCE_SURFACE; returns persisted:false, scope:'session' and the new tools' schemas). A widenable stdio session's catalog lists the full surface. {tools} returns each visible op's registry schema, so a starter client also gets the fullSurfaceOnly params its tools/list leaves out (dispatch accepts them on every surface). {surface, tools} together (GBRA-449) runs the {surface} branch first, exactly as alone (refusals included), then the {tools} branch against the resulting surface, and answers the descriptor shape plus surface_change (the surface answer). Tests: test/request-tools.test.ts, test/request-tools-stdio.test.ts, test/request-tools-combined-gbra449.test.ts.

  • src/mcp/surface.ts — MCP tool-surface modes: 'verbs' (exactly the ops marked verb: true) | 'starter' (the ~27-op daily-driver set) | 'full' (default — identity; existing installs unchanged). STARTER_OPS is composed PROGRAMMATICALLY: a spread of VERB_NAMES (never a hand-count) + the fallback daily slice (BRAIN_TOOL_ALLOWLIST + the agent lane submit_agent/get_agent_job) + whoami + request_tools + capture (a DIRECT literal, deliberately NOT via the allowlist); re-derived from production usage via scripts/derive-starter-ops.ts (paste-in proposal, never an auto-edit). ALWAYS_INCLUDED_STARTER_OPS (verbs + whoami + request_tools + the agent lane + capture) is the exported always-set consumed by BOTH derive-starter-ops.ts and the advisor starter-fit collector, so neither re-types the composition. parseSurfaceFlag (strict, loud reject; reads --surface v and --surface=v through flagValues in src/core/direct-pool-profile.ts, and a repeated flag takes the narrowest), resolveSurface/resolveSurfaceWithSource (flag > config mcp_surface > full, the serve --http order), resolveStdioSurface (stdio: GBRAIN_SURFACE > flag > config > full; an invalid env value is returned as invalidEnv, never fatal; surfaceEnvInvalidNotice builds its notice), SURFACE_SOURCE_LABEL, filterOpsForSurface, allowedOpNames, createStdioSurfaceState (the stdio session's one mutable allow-set; widen(to) recomputes it, read-only stays read-only), sessionWidenAllowed (mcp.allow_session_widen, DB > file, default true). STARTER_OPS also carries mute_notice, the dismissal for the coaching notices starter sessions receive. Enforcement is TWO-layer and fail-closed: the advertised list AND dispatchToolCall's allowedOps set (a hidden op returns unknown_tool even when called by name) — applied on stdio (server.ts) and BOTH HTTP paths (serve-http.ts after !localOnly, and the second transport http-transport.ts). Per-client ceiling mac... (line truncated to 2000 chars)

  • src/mcp/tool-catalog.ts — renderToolCatalogMarkdown(): the docs/TOOL_CATALOG.md renderer. Config-independent + deterministic (no engine/config reads, no timestamps): non-localOnly ops grouped one section per Operation.area, per-op first-sentence description (from buildToolDefs's non-strict shape), scope, STARTER_OPS membership, publish-gate key. Generated by scripts/generate-tool-catalog.ts; freshness-guarded by scripts/check-tool-catalog-fresh.sh in bun run verify (the METRIC_GLOSSARY pattern). Pinned by test/tool-catalog.test.ts.

  • src/mcp/tool-defs.ts — buildToolDefs(ops, opts?) helper; the stdio MCP server, the OAuth HTTP tools/list, and the subagent tool registry all consume it, byte-for-byte equivalence pinned by test/mcp-tool-defs.test.ts. opts.strictParams: true (when mcp.strict_params resolves 'reject') additionally declares the _meta/dry_run/room passthrough keys in properties and closes each schema with additionalProperties: false (schema-validating clients must not strip _meta.session_id); both emission states pinned. Exports the recursive paramDefToSchema(p: ParamDef) — single source of truth for ParamDef→JSON Schema mapping shared by buildToolDefs and src/core/minions/tools/brain-allowlist.ts (subagent registry). Recursive on items so nested array-of-arrays preserves inner shape on the wire, and on object properties (closed with additionalProperties: false, member required keys). Key ordering (type, description, enum, default, items, properties) is intentional so JSON.stringify output stays byte-stable. toolAnnotations(op) is the one annotation rule: curated op annotations win; else readOnlyHint: true for a read-scope op tagged mutating: false, readOnlyHint: false for one tagged mutating: true, nothing for untagged ops. test/mcp-tool-defs.test.ts has a findArrayWithoutItems walker that fails on any type: 'array' lacking items.type. Annotations: curated op.annotations win; else readOnlyHint: true iff mutating === false, readOnlyHint: false (+ idempotentHint: true when idempotent) iff mutating === true; every op declares both tags (test/ops-mutation-tags.test.ts).

  • src/mcp/validate-params.ts — param normalization + validation for dispatch (direct unit surface). validateParams (required/type/enum — enum membership is a TYPE error in BOTH strict modes; the caller's raw value is never echoed into the message), normalizeOptionalParams (null/'' optional-param idioms become truly absent, and so do the values a param's schema rules out: 0 below minimum/exclusiveMinimum, [] under minItems, {} on an object with a required member, false on an actsWhenTrue flag, then any param whose appliesWith param is absent (GST-927); required params untouched, copy-on-write), findUnknownParams (unknown top-level keys on the NORMALIZED object; UNKNOWN_PARAM_ALLOWLIST = _meta + dry_run; per-op did-you-mean from the op's OWN declared params only, plus the explicit PARAM_ALIAS_HINTS aliases edit distance cannot reach, e.g. add_timeline_entry content → detail and remember facts → items, text → fact; dispatch rejects unknown keys on every mutating op regardless of mcp.strict_params, so a typo never commits a write, and appends the op's optional callShape (a valid call with placeholder values, set only by remember, GST-1493) to the suggestion), buildUnknownParamWarnBlock (the model-visible warn notice), resolveStrictParamsMode(engine, config) (dual-plane mcp.strict_params, DB > file > 'warn'; a failed DB read applies the LAST successfully read DB mode, held per engine — fail-closed, so a transient config outage on a reject-mode server cannot re-open the warn grace period; test seam resetStrictParamsModeCache()). Privacy: raw unknown key names reach the CALLER only, never mcp_request_log. Pinned by test/validate-params.test.ts. Every path that hands arguments to an op handler runs normalize → validate first: dispatchToolCall, handleToolCall (gbrain call) and the SkillOpt rollout and write-capture registries, so a model that sends "" or null for every optional argument reads t... (line truncated to 2000 chars)

  • src/core/mcp-request-log-retention.ts — opt-in request-log retention (cycle.purge.mcp_request_log_retention_days: unset/0/off = off, 30-3650; mcpLogRetentionRefusal is the config set refusal). pruneMcpRequestLog runs batches of 2,000 (200 per run, the cutoff fixed once), each ONE statement: oldest rows past the cutoff in created_at order FOR UPDATE SKIP LOCKED, minus AUDIT_LOG_OPERATIONS, deleted and added to mcp_request_log_rollup (requests, last_used_at, pruned_through = newest pruned row; NULL token under ''), so a failed tally deletes nothing; a missing tally table skips. runMcpLogRetention reads the key and records {at, deleted, capped} under cycle.purge.mcp_request_log_last_prune; callers: the cycle purge phase and startMcpLogPruneTicker (PGLite only, hourly, 3 s budget, yields between batches) in both serve lanes. lifetimeUsage = kept rows + tally in one statement (admin Agents totals). Pinned by test/mcp-request-log-retention.test.ts (both engines) and test/serve-http-admin-request-log-retention.test.ts.

  • src/commands/doctor/checks/mcp-request-log.ts — doctor mcp_request_log_retention (ops): log rows (reltuples on Postgres, a bounded count on PGLite) and size; retention off past 1M rows or 1 GiB warns with an ask_user + destructive fix (config set … 90), never a remediation step; retention on reports pruned_before and the last run, and warns when no prune ran for 48 hours (or none ever ran while rows past the window wait). Pinned by test/doctor-mcp-request-log-retention.test.ts.