Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript
50.4 KB
Raw Blame

Key files: Core Ai

Subsystem index. Read only the entries relevant to your change. Current behavior and load-bearing invariants; history belongs in Git and CHANGELOG.

  • src/core/ai/build-gateway-config.ts — buildGatewayConfig(c: GBrainConfig): AIGatewayConfig, re-exported by src/cli.ts. Lets core modules (init-embed-check.ts) reuse it without importing the CLI entrypoint. Delegates the file-plane API-key fold + env merge to mergedProviderEnv (src/core/ai/provider-env.ts); keeps ownership of threading local-server *_BASE_URL env vars into base_urls. process.env wins EXCEPT empty-string / undefined values are dropped before the merge, so an injected empty ANTHROPIC_API_KEY='' (Claude Code neuters subprocess LLM calls this way) can't clobber a valid config-plane key; '0' / 'false' are preserved. Pinned by test/ai/build-gateway-config.test.ts.

  • src/core/ai/call-log.ts + the observer half of src/core/ai/invocation-guard.ts — invocation observers and attribution. observeAIInvocations(observer) registers a process-level observer that invokeAI notifies once per call (guarded or not) with { call, outcome: ok | error | refused, usage, startedAt, ms, attribution }; the unguarded branch of guarded-generation.ts routes through invokeAI only while an observer is installed and passes the caller's transport options unchanged, so observation never alters retries, token defaults or admission (pinned by test/ai-invocation-observer.test.ts). Observer throws are swallowed and counted (aiInvocationObserverFailures); adding the same function twice is a no-op. withAIAttribution({ request_id, effect, job_id, job_name, phase }, fn) merges into an async-local attribution read by observers: the persistence effect dispatcher (effects.ts), the in-process facts queue (facts/queue.ts, captured per entry at enqueue), the facts-absorb job handler (internal write-request id from persistence_request_id) and timeContainedPhase (cycle:<phase>) set it. installAICallLogFromEnv (called by configureGateway, idempotent per path) appends one JSON line per call to GBRAIN_AI_CALL_LOG (kind with multimodal counted as embedding, operation, model, outcome, ms, input/output/cache tokens or unknown, attribution; never prompt or response text; one warning on write failure). Pinned by test/ai-call-log.test.ts.

  • src/core/ai/chat-usage.ts + src/core/ops/usage.ts — durable per-call chat usage accounting. gateway.chat() calls recordChatUsage at its SUCCESS boundary (production provider path AND test-transport path) with the answering model + token usage, and gateway.expand() does too on each successful expansion call with phase expansion, which wins over an ambient phase (#7176); think's synthesis call runs under withDefaultChatPhase('think'), so it records think unless a job or cycle phase already attributes it; the record lands in the chat_usage_log table (migration v140) via an injected sink (setChatUsageSink — no engine import here, so the gateway imports this leaf statically without deepening a cycle). Phase attribution is best-effort via AsyncLocalStorage (withChatPhase): the minion worker AND the cycle's inline drain (inline-drain.ts) both wrap handler execution in job:<name>, a cycle phase that meters its own spend wraps itself in phase:<name> (dream synthesize does), and direct callers record phase NULL. The innermost wrap wins, which is the point — a child drained INSIDE a wrapped phase must carry its own job: tag or the phase tag absorbs every child's spend and the ledger double-counts (one ledger per surface: minion_jobs stays the child-spend authority, the phase row is the orchestrator's own calls). Pricing resolves through CANONICAL_PRICING (estimateChatCostUsd, cache_read/cache_write at provider cache rates when the table carries them); unknown models record cost_usd = NULL, never a fake 0. Accounting is strictly fail-open + fire-and-forget: a sink error must never break a chat call. withChatCallMeter counts successful calls on a ChatCallMeter (phase containment's { calls }); a meter that declares cost_usd (the nightly probe's) also prices each call with the budget tracker's resolver (priceFor with the meter's pricing_overrides; a table row uses the same cache-aware estimateChatCostUsd value the record carries), counts an unpriced call in unpriced_calls instead of as 0, and is priced even when no sink is registered. The get_usage op (admin scope, NOT read — chat_usage_log has no source dimension, so a source-restricted/federated token must not see brain-wide spend telemetry) returns per-model/per-phase aggregates only, plus an explicit coverage block stating what the ledger does NOT capture (subagent raw-SDK path, embeddings, pre-sink/pre-v140 calls, failed calls); budget-tracker.ts remains the pessimistic in-flight spend gate — this table is the after-the-fact ledger.

  • src/core/ai/date-grounding.ts — the time vocabulary every extraction prompt shares: observation time (when text was written or said), event/validity time (facts.valid_from), recording time (created_at). resolveObservationDate (filename / date / published / created keys via computeEffectiveDate; never event_date, row timestamps or now), observationDateFrom (turn/message timestamps), observationDateRule (static system-prompt rule: rewrite relative dates as absolute dates or bounds resolved against the observation date, never leaving the phrase beside its date; unknown → keep as written), observationDateLine (user-message date or unknown), parseExtractedEventDate (YYYY-MM-DD, 1900..now+1y), resolveValidFrom (extracted > caller > observation > now), hasUnresolvedRelativeDate (judge-free eval check). Fact extraction, dream synthesis, extract_atoms and propose_takes add the rule by default (extraction.date_grounding unset or true; false turns all off; held-out verdict and per-consumer checks in docs/eval/decisions/p2-e2-heldout/); the life chronicle judge adds it only when the setting is explicitly true (isConsumerDateGroundingOn(engine, consumer)). Page-body facts default to the page's observation date regardless. Pinned by test/ai/date-grounding.test.ts, test/facts/extract-date-grounding.test.ts, test/facts/backstop-page-date.test.ts.

  • src/core/ai/defaults.ts — leaf module for canonical embedding and reranker defaults, without a gateway import. Voyage voyage:voyage-4 at 1024 dimensions is the new-install embedding default; voyage:rerank-3-lite is the runtime/mode-bundle reranker default (it applies only to brains with no search.reranker.model row). renderCanonicalMigrationCommands supplies an explicit-target cost preview and an OpenAI keep-width alternative where supported. These constants do not establish the identity of vectors in an existing database and must never authorize automatic provider conversion. Init preserves explicit choices, writes no reranker row when the Voyage key is available, disables reranking for keyed installs without that key, and preserves keyless setup.

  • src/core/ai/dims.ts — per-provider providerOptions resolver for embed-time dimension passthrough; the single source of truth for "which provider needs which knob to produce vector(N)". Exports dimsProviderOptions(implementation, modelId, dims) (called by embed() in gateway.ts), VOYAGE_OUTPUT_DIMENSION_MODELS (private const — the 7 hosted Voyage models that accept output_dimension: voyage-4-large, voyage-4, voyage-4-lite, voyage-3-large, voyage-3.5, voyage-3.5-lite, voyage-code-3 — nano deliberately excluded), VOYAGE_VALID_OUTPUT_DIMS = [256, 512, 1024, 2048] as const, supportsVoyageOutputDimension(modelId), isValidVoyageOutputDim(dims). Voyage path uses the SDK-supported dimensions field ({ openaiCompatible: { dimensions: N } }), NOT Voyage's output_dimension wire-key — the voyageCompatFetch shim in gateway.ts:541 translates dimensions → output_dimension before the HTTP body is built (the AI SDK's openai-compatible adapter doesn't recognize the wire-key, so sending it from here would be silently dropped and Voyage would return its default 1024-dim). Runtime guard: when a Voyage flexible-dim model is configured with dims outside VOYAGE_VALID_OUTPUT_DIMS, throws AIConfigError with a paste-ready gbrain config set embedding_dimensions <256|512|1024|2048> hint at the embed boundary (most common trigger: embedding_model: voyage:voyage-4-large without embedding_dimensions, retaining an incompatible 1536-dimensional setting from another provider). Every lookup in this module folds through the private modelMatchKey (trim + lowercase) so cased hub-form ids (Qwen/Qwen3-Embedding-4B) match the all-lowercase tables — the folded key is a MATCH key only, never sent to a provider (wire model ids are case-sensitive; error messages keep the original id so they stay paste-ready); the Qwen3-Embedding native-width table lives at module scope alongside the other dim tables. Consequence, deliberate: a cased Voyage/Perplexity config matches and fails loudly at init when its dims is invalid, rather than skipping validation and silently producing wrong-width vectors.

  • src/core/ai/embed-batch-plan.ts — the embedding request plan shared by embed() and spend preflights: per-input truncation (EMBED_MAX_CHARS), the max_batch_tokens × safety-factor pre-split (splitByTokenBudget, re-exported by the gateway), the item cap (capBatchItems, NO_BATCH_CAP_SUB_BATCH_ITEMS for no_batch_cap recipes) and the per-request input ceiling a guard reserves (embedRequestMaxInputTokens: one token per UTF-8 byte, capped per text by a declared max_input_tokens, additive so any split or retry of the same texts stays inside it). embedRequestCeilings(texts, model) replays the plan for a model string without a gateway; rerankRequestMaxInputTokens is the rerank twin. Pinned by test/embedding-migration-settle.serial.test.ts and the adaptive-batch suites.

  • src/core/ai/embedding-guard.ts — the embedding refusal contract (#4616). degenerateEmbeddingReason flags zero-norm (L2 norm below EMBEDDING_NORM_EPSILON), NaN and ±Inf vectors; sendableEmbeddingInputs keeps empty and whitespace-only inputs away from the provider; screenEmbeddings / screenAlignedEmbeddings put vectors back at their input positions and throw EmbeddingZeroNormError (an AIConfigError, code: embedding_zero_norm, docs docs/guides/write-refusals.md#embedding_zero_norm) carrying per-item failures and the usable vectors aligned with the input (null where refused). mergeZeroNormErrors re-aligns per-slice refusals. suggestionFor(slug, sourceId?) fills the gbrain embed <slug> recovery command. fitMatryoshkaPrefix(recipe, modelId, vector, expected) (PR #5921, wave 14): when the recipe's embedding touchpoint declares the model in matryoshka ({ native, prefixes }, folded ids; recipes/nvidia.ts declares Nemotron-3-Embed-8B at 4096 with 1024/2048) and the provider answered at the native width for a listed target, the vector is sliced from the start and L2-normalized before gateway.embedSubBatch's dim-mismatch check; every other vector returns untouched, an overflowing norm is refused, a zero or non-finite prefix falls to the per-item screen. Consumers: gateway.embed() and both multimodal embedders, embedding.ts:embedBatch (later 100-input slices still run), embed-retry.ts (never retried; embedBatchKeepingUsable returns the partial result), embed.ts:embedPageTexts and markdown/code imports (usable vectors stored, refused chunks stay NULL and the page signature is not stamped). Tests: test/embedding-zero-norm-4616.serial.test.ts (+ Postgres wrapper).

  • src/core/ai/engine-gateway.ts + src/core/engine-scope.ts (GBRA-283 B2) — the AI gateway of an engine openEngine opened. engineGatewayInput(settings) builds its AIGatewayConfig from settings.ai (provider id → apiKey/baseUrl, folded into the recipe's key env name and base_urls or the native *_BASE_URL) and settings.embedding, never process.env or ~/.gbrain/config.json; scopedGatewayState keeps one state (config, model clients, a query-cache tag, batch-shrink, no-key and structured-output memory) per held settings object, so a disconnect drops it; settings.cache_namespace tags that engine's cached query embeddings instead (B3a). engineProviderProblem refuses claude-cli and recipes with resolveOpenAICompatConfig (Azure). gateway.ts reads gw() (that state in an embedded engine's scope, else the process state configureGateway writes) and configFor(recipe, touchpoint) refuses engine_scope_missing before any call to a provider the engine lacks; a call with no scope while the process holds an open embedded engine refuses engine_scope_required (refuseUnscoped) whatever configureGateway set; discovery and configureGatewayIfUninitialized skip engine states, and anthropic-key.ts answers the scoped engine's own key. withEngineScope(engine, fn) (AsyncLocalStorage, engine-scope-store.ts; entered even before the first embedded engine opens) is entered by every operation handler (operation-load.ts), each persistence consumer (persistence/service.ts), minion jobs, POST /mcp, and hybridSearch and the import paths (needsEngineScope: once an embedded engine has opened, any engine other than the scoped one); withLocalScope(fn) marks a call for the process gateway (the CLI, serve and jobs enter it); embeddedEngineInScope resolves through engineKey. engine-construct.ts builds an engine without the process chat-usage sink (createEngine adds it), so openEngine registers a sink of its own. Pinned by test/engine-gateway.test.ts and test/engine-scope-required.test.ts (refusals, two engines, transaction/proxy/degraded sharing, a byte-identical process gateway for other engines, a canary key absent from errors, logs, doctor and read-back).

  • src/core/ai/errors.ts — error normalization and classification. One private generator, wrappedErrorLayers, opens a thrown error's wrappers outermost first (cause, else RetryError's lastError; objects only, at most five layers, a repeated layer ends the walk). isStructuredOutputRejection and readProviderFailureSignals use it; the latter returns the first finite status/statusCode/apiErrorStatus plus every string name and code and never throws (a throwing getter ends the walk). page-summary.ts classifyChatError reads those signals, so a wrapped claude-cli 429, an AI SDK RetryError's 429/5xx or a TimeoutError/network code on cause classifies as rate_limit/provider_5xx/timeout/network instead of malformed (which drops the page to the title tier). classifyGlobalLlmError keeps its own depth-3 loop over cause ?? lastError, so a 404 there halts as model_not_found; the model-not-found phrases (model_not_found, not_found_error, "does not exist or you do not have access", "does not have access to [the] model") are tested before the status map, so a provider 403 that denies a project access to a model classifies model_not_found with the model fix, never auth (a plain 401/403 stays auth; test/ai/model-not-found-class.test.ts). #5304: normalizeAIError(err, context, redact, routeHint) appends routeHint to a 404's fix only; src/core/ai/model-not-found-hint.ts renders it from the gateway's recorded provenance (gatewayModelNotFoundHint(touchpoint, model) → modelRouteHint: "Selected via models.tier.reasoning. Fix: gbrain config set models.tier.reasoning :"; env and config.json pins say what to clear; a deprecated key names the models.* key to set instead; the built-in default names the touchpoint's tier key; unrecorded or stale provenance adds nothing) and embeddingModelNotFoundHint() names embedding_model with the doctor route, never a bare config set. gateway.ts passes it at its embed, expand and chat 404 sites (test/ai/model-not-found-hint-5304.test.ts); the --- raw --- phrase barrier and billing-over-429 precedence hold (test/synopsis-chat-failure-class.test.ts, test/ai/classify-global-llm-error.test.ts). #6816: providerContentBlockReason also returns claude_cli_usage_policy for the claude-cli adapter's usage-policy refusal (ClaudeCliProcessError, no apiErrorStatus, claude-cli reported error: API Error: prefix, the AUP link and a refusal phrase: CLAUDE_CLI_REFUSAL_RE); test/claude-cli-refusal-6816.test.ts. #9466: normalizeAIError turns a success response whose body the AI SDK could not parse as one JSON document (AI_JSONParseError, alone or as the cause of a 2xx AI_APICallError) into a non-retryable AIConfigError with code provider_response_unparseable (UNPARSEABLE_RESPONSE_CODE) and fixed text that quotes no response body; facts/extract.ts puts that code in its cause label and facts/dead-absorb-retry.ts classifies it not_retryable (test/ai/unparseable-success-response-9466.test.ts).

  • src/core/ai/gateway.ts — unified seam for every AI call. The ai SDK loads at the first provider call (lazy default transports), not on configure. embedQuery(text, opts?) and isAvailable(touchpoint, modelOverride?) accept a model override so the resolved-column path embeds via the column's provider (Voyage / OpenAI) instead of the global default; the hybrid path passes {embeddingModel: resolved.provider, dimensions: resolved.dimensions} and the gateway resolves the matching recipe via instantiateEmbedding(). isAvailable('embedding', 'voyage:voyage-3-large') checks the override's recipe (not the default) so hybrid skips vector search only when the active column's provider is actually down. voyageCompatFetch adapts Voyage embedding requests and responses; openAICompatAsymmetricFetch threads explicit query/document input types for other compatible recipes through the SDK boundary. gateway.rerank() native HTTP path (no AI-SDK reranking abstraction): resolves the EFFECTIVE reranker as input.model ?? getRerankerModel() ?? DEFAULT_RERANKER_MODEL (imported from ai/defaults.ts; voyage:rerank-2.5), posts to the recipe's reranker path (touchpoints.reranker.path, default /rerank; supported recipes declare their path explicitly) with bearer auth — the request's top-N key is recipe-pluggable via touchpoints.reranker.top_param (default top_n; Voyage top_k) — and returns RerankResult[] sorted by relevance. no_key preflight: after requireConfig(), a recipe without a custom resolveAuth whose auth_env.required key is absent from cfg.env throws RerankError('no_key') BEFORE any HTTP — noKeyOnce() writes ONE no_key audit row per process per model without a stderr line, avoiding per-query noise. auth therefore means "key present but rejected" (HTTP 401/403). RerankError.reason classifier: auth | no_key | rate_limit | network | timeout | payload_too_large | unknown. 5s default timeout (search hot path). Pre-flight payload guard rejects bodies over recipe.touchpoints.reranker.max_payload_bytes with reason: 'payload_too_large'. _rerankTransport test seam mirrors _embedTransport. embedQuery(text) threads inputType: 'query' through dimsProviderOptions() (4-arg). getRerankerModel() returns only the EXPLICITLY configured model (callers need not pre-check availability — rerank() fails no_key itself; the sync readiness predicate for dashboards is reranker-readiness.ts) + isAvailable('reranker') branch; configureGateway + reconfigureGatewayWithEngine thread reranker_model; applyResolveAuth + defaultResolveAuth widen touchpoint param to include 'reranker'. embedMultimodalOpenAICompat() routes recipes with implementation: 'openai-compatible' (LiteLLM, Anyscale, vLLM, Gemini multimodal via proxy) through the standard /embeddings endpoint with content arrays carrying image_url entries; the Voyage /multimodalembeddings path is unchanged (gateway selects by recipe implementation tag). Runtime dimension validation throws AIConfigError (with model id + observed + expected) before the vector reaches storage when the provider returns a width that doesn't match the recipe's default_dims or the brain's embedding_dimensions. Pinned by test/openai-compat-multimodal.test.ts. Module-scoped _embedTransport defaults to AI SDK embedMany, with __setEmbedTransportForTests(fn) test seam so tests drive embed() with a stubbed transport. splitByTokenBudget and isTokenLimitError exported @internal (pure functions reused by the test file). Module-level _shrinkState: Map<recipeId, {factor, consecutiveSuccesses}> halves the recipe's effective safety_factor on token-limit miss (floor 0.05) and heals back ×1.5 after SHRINK_HEAL_AFTER=10 consecutive successes. configureGateway() walks every registered recipe at construction and emits a once-per-process stderr warning for any embedding touchpoint missing max_batch_tokens (excluding the canonical OpenAI fast-path). resetGateway() clears _shrinkState, the warned-set, and restores the real transport. embedMultimodal() reads cfg.embedding_multimodal_model first (falls back to cfg.embedding_model); after the recipe-level supports_multimodal fast-fail, validates the resolved model against touchpoint.multimodal_models when declared (closes the Voyage-text-only-into-multimodal-endpoint footgun before any HTTP call). getMultimodalModel() accessor mirrors getEmbeddingModel / getChatModel. src/core/ai/multimodal-model.ts:multimodalEmbeddingModel() answers the model embedMultimodal() would use when it passes those same recipe and allow-list checks (null otherwise, or when the gateway is unconfigured); hybrid search routes an inferred image intent and the LLM modality tie-break to the image arm only when it is set (test/cross-modal-hybrid-integration.serial.test.ts). Exported VoyageResponseTooLargeError tagged class: voyageCompatFetch's two OOM-defense caps (Layer 1 Content-Length, Layer 2 per-embedding base64) throw it; the inbound response-rewriter's try/catch (which swallows parse failures so misshaped responses fall through to the SDK parser) checks instanceof VoyageResponseTooLargeError and rethrows so the cap is actually effective (test/ai/gateway.test.ts drives both layers through embed()). AI SDK v6 toolLoop compat (gbrain skillopt rollouts AND production background subagent jobs both route through chat() / toolLoop): in chat(), tool defs wrap the raw JSON Schema with the SDK's jsonSchema() helper (inputSchema: jsonSchema(t.inputSchema)) — v6's asSchema() treats a bare {jsonSchema: ...} object as a thunk and throws "schema is not a function"; exported pure toModelMessages(messages: ChatMessage[]): unknown[] converts gbrain's provider-neutral ChatMessage[] into v6 ModelMessage[] — tool results (pushed by toolLoop as role:'user' with bare-value tool-result blocks) become a dedicated role:'tool' message with structured output:{type:'json'|'text'|'error-text', value} parts; null output preserved as {type:'json', value:null} (not dropped); text/tool-call blocks pass through with v6 field names (toolCallId/toolName/input); applied at the generateText call (messages: toModelMessages(opts.messages)). The converter is load-bearing for the production subagent path, not just skillopt. Pinned by test/gateway-model-messages.test.ts. Companion: src/core/skillopt/rollout.ts builds tool schemas through the shared paramDefToSchema from src/mcp/tool-defs.ts (single source of truth, recursive on items/enum/default), never an inline converter. Provider-agnostic plumbing: resolveNativeBaseUrl(provider, cfg) normalizes a configured ANTHROPIC_BASE_URL / OPENAI_BASE_URL to carry the /v1 suffix and is passed explicitly at every native createAnthropic / createOpenAI site (chat/expansion/embedding), so an env-injected bare host doesn't 404; returns undefined when unset so the SDK default is preserved (Google too: docs/ai-providers/google.md). diagnoseEmbedding fails closed with user_provided_dims_unset when a user-provided / zero-default recipe (litellm/llama-server) has no configured embedding_dimensions. configureGateway does not backfill embedding_dimensions (readers default it themselves), keeping the "no dims set" signal honest for that guard and the multimodal skip. withBudgetTracker: gateway-layer enforcement via AsyncLocalStorage<BudgetTracker>. withBudgetTracker(tracker, fn) installs the tracker on the module-internal store; every gateway.chat / embed / rerank call inside the scope auto-composes (reserve before, record in try/finally). Outside-scope calls are budget no-ops. Nested scopes restore the outer tracker on exit. getCurrentBudgetTracker() is the test seam. The chat path uses the pessimistic fallback on error paths; the embed path estimates input tokens from char count × recipe's chars_per_token because the AI SDK doesn't surface per-batch embed token usage; the rerank path estimates char count of query+docs. Pinned by 6 unit cases. reconfigureGatewayWithEngine(engine) (async, called from cli.ts after engine.connect(), before every command except CLI_ONLY no-DB commands) re-resolves expansion + chat defaults through resolveModel() so models.tier.* and models.default overrides apply to both. DEFAULT_CHAT_MODEL is anthropic:claude-sonnet-4-6. ChatOpts.timeoutMs replaces the default chat backstop (GBRAIN_AI_CHAT_TIMEOUT_MS, 300 s) for one call and ToolLoopOpts.turnTimeoutMs forwards it to every turn (the subagent handler's per-turn cap, #4921); ChatOpts.thinking: 'off' maps per route through thinking-off.ts (#5331). ChatOpts.temperature is threaded verbatim to the AI SDK generateText call (unset → the provider's default; the LongMemEval judge pins 0, the official evaluate_qa.py setting). ChatResult.responseModel carries the model id the PROVIDER reported (response.modelId, e.g. a dated snapshot) when the SDK surfaced one and is absent otherwise, while model stays the requested provider:modelId; eval receipts pin the two side by side (reader_model_snapshot, judge_model_snapshot). __setChatTransportForTests mirrors __setEmbedTransportForTests so tests drive chat() with a stubbed transport. toolLoop per-turn permit hook: optional acquireTurnPermit() acquires a provider permit before EVERY round-trip and releases in finally; a lease-full throw propagates without consuming the turn (the subagent path wires it to the rate leases with provider-derived keys — Anthropic models share anthropic:messages, others get <recipeId>:chat). ToolLoopStopReason includes 'length': a zero-tool-call turn that hit the output cap propagates 'length' instead of folding into 'end', so truncation is never reported as a clean finish; and 'empty_response' (#6448): a zero-tool-call clean stop whose text is empty or whitespace-only, so a refusal body or thinking-only turn is never reported as a finished answer (the subagent handler maps it to error). Per-part provider state: ChatBlock variants carry optional providerMetadata captured from SDK parts in chat() and re-emitted as providerOptions on the rebuilt parts in toModelMessages() (attached only when present — metadata-free blocks stay byte-identical); this is the Gemini 3.x thoughtSignature echo. isThinkingByDefaultModel(modelStr) (exported) matches Claude 5-family ids behind any provider-prefix chain with a letters-only family segment (never claude-3-5-*); isThinkingModel(modelStr) (exported) is that regex OR the recipe's chat thinking_by_default capability (fail-closed: unknown/chat-less recipes are non-thinking) and drives defaultMaxOutputTokens's 32k thinking headroom for chat()/toolLoop() callers that omit maxTokens (e.g. gbrain skillopt rollouts) and the subagent handler's resolveMaxOutputTokens; think/index.ts shares the regex and makes the same capability check. Pinned by test/ai/gateway-thinking-headroom.test.ts. expand() and generateOcrText() also record on the ambient tracker (they call generateObject/generateText directly and never pass through chat()'s _recordBudget): record-only, no reserve — a breach surfaces on the NEXT reserving call, matching chat()'s swallow of BudgetExhausted from record(). Successes record normalized SDK usage via normalizeSdkUsage (the ONE home for the v6/legacy usage shapes, used by chat's success path too; first FINITE field wins so a NaN v6 field can't shadow a real legacy value or poison the running total); failures record pessimistically under gateway.expand.failed / gateway.ocr.failed (a rejected attempt still billed provider tokens — one expand() can legitimately produce two records when the structured-output attempt fails and the text fallback runs). A recipe whose declared structured-output support is rejected at call time is remembered for the process lifetime (_structuredOutputRejectedRecipes) so the rejected attempt isn't re-paid on every call. OCR's input estimate is prompt text + a fixed per-image token constant, never base64 length (bytes are not tokens). generateOcrText() routes to getImageOcrModel() — embedding_image_ocr_model when set, else the expansion model — gated by isAvailable('expansion', <ocr model>) so a misconfigured OCR model (provider without an expansion touchpoint, or unkeyed) fails closed to '' instead of silently OCRing with the expansion model; the accessor mirrors getMultimodalModel(), configureGateway/buildGatewayConfig thread the field, and an unconfigured gateway stays a silent '' no-op. Pinned by test/ai/ocr-model-routing.test.ts. __setGenerateObjectTransportForTests mirrors the generateText seam. ChatOpts.responseSchema ({ name, description?, schema }) is honored ONLY when the resolved recipe is openai-compatible AND declares supports_structured_outputs: chat() attaches a tolerant Output spec (jsonSchemaOutput — carries the { type: 'json', schema, name } responseFormat the provider turns into response_format: json_schema, and parseCompleteOutput hands the raw text back instead of throwing NoObjectGeneratedError the way Output.object does on unparseable text) to the generateText call, so result.text, stopReason mapping, budget recording and the caller's own parse + retry lane run unchanged; native / claude-cli lanes and undeclared openai-compatible recipes never see the field. A declared recipe whose backend REJECTS the schema at call time (isStructuredOutputRejection in errors.ts: a request-shaped 4xx naming response_format / json_schema / structured output — never a 5xx or an unrelated 400) gets ONE schemaless retry plus one warn line, and lands in _structuredOutputRejectedRecipes so later chat() and expand() calls skip the schema for the process lifetime instead of re-paying the rejection on every retry lane. + test/core/budget/expand-records-budget.test.ts + test/ai/gateway-ocr-budget.test.ts + test/facts-extract-structured-output.test.ts Tracker recording goes through src/core/ai/budget-record.ts:recordOnTracker (fail-open: a cap breach on record() surfaces on the next reserve()); every chat/toolLoop/expand/embed/rerank/OCR record site passes the requested model, purpose (optional on ChatOpts + toolLoop opts; skillopt sets skillopt.optimizer|target|judge, engine-internal calls leave it null), failed on failure paths, countsAsCall: false for expand's structured-output fallback attempt and estimated when tokens are char-estimated. reconfigureGatewayWithEngine records the expansion/chat ResolveSource (file_config for the servable config.json pin) in src/core/ai/gateway-model-sources.ts; its getters answer only while the gateway still routes to the recorded model, so a later reconfigure reads as unknown instead of stale provenance. embedQuery caches query vectors per process (src/core/ai/query-embed-cache.ts: 512-entry LRU, 10-min TTL), keyed by recipe, model id, base_urls override, effective dims and the exact provider input (prefix + text). It stores only resolved vectors, as copies; the cache clears with _modelCache (configure, env refresh, engine re-stamp, reset, __setEmbedTransportForTests) and a generation counter drops an embed straddling a clear; an embed whose own engine closed mid-flight stores nothing. Document-side embed() is never cached; a caller under an AI invocation guard skips the cache read. Prompt-cache breakpoints: cacheSystem: true on a prompt-cache-capable route derives ONE cacheControlValue (config TTL override honoured) that marks the system block, the last tool def, the call-level auto marker and, since PR #6341, any user text ChatBlock flagged cache: true (toModelMessages(messages, cacheControlValue) merges providerOptions.anthropic.cacheControl onto that block); without a value the flag is dropped and the block is plain text, so non-caching routes and the claude-cli adapter (which joins text parts with \n) see the same prompt text. Pinned by test/page-summary-prompt-cache.test.ts.

  • src/core/ai/chat-fallback.ts: the chat_fallback_chain consumer. gateway.chat() hands a call to chatWithFallback when a chain is configured and the caller did not pass allowFallback: false; configureGateway normalizes the chain with normalizeChatFallbackChain (a comma-separated string, the env and DB form, is split). Attempts run the call's own model, then each chain entry not already tried (chatFallbackAttempts), each as a complete single-model chat(), so budget, usage ledger, guardrails and ChatResult.model follow the model that ran; an answer from a chain entry also carries ChatResult.fallbackFrom (the call's own model). A provider error (a claude-cli subscription limit arrives as apiErrorStatus 429) or a structural refusal (stopReason refusal/content_filter) moves to the next model with one [ai.gateway] stderr line, and the first hop of a process queues one chat_fallback_hop safety notice in fallback-hop-queue.ts; with chat_fallback_on_refusal: false (ChatFallbackOptions.onRefusal, passed by chat() from the gateway config) a structural refusal is returned and a provider content block (providerContentBlockReason: a Gemini prompt block, or an HTTP 400 content-policy body such as OpenAI invalid_prompt, Azure content_filter or DeepSeek "Content Exists Risk") is thrown instead of being sent on, while provider errors still fall back; a policy refusal (isAIInvocationPolicyError or a BudgetTracker BudgetExhausted, including an unpriced chain entry under an explicit cap) or the caller's own abort stops the chain, and a cancel that lands while an attempt refuses returns that refusal instead of forwarding it; an earlier refusal is returned in preference to a later error, otherwise the call's own model's error is thrown (the last failure logged), so retry and halt classification follow the configured model. toolLoop forwards allowFallback to every turn. Judge, critic and eval call sites pin their model: brainstorm judges, grade_takes, drift, the edge_contradictions judge (apply mode defaults from a per-model certification), voice-gate, eval-contradictions, notability-eval, takes-quality-eval, cross-modal eval, skillopt (judge, optimizer, rollouts), the LongMemEval reader and judge, the functional-area-resolver harness, the models doctor / providers test probes, decide() (its egress gate cleared one provider), auto_think (its BudgetMeter priced one model), and think with an explicit --model; the synthesize triage and concept synthesis fall back. Pinned by test/ai/gateway-chat-fallback.test.ts and test/ai/chat-fallback-on-refusal.test.ts (refusal switch through the real loader on every plane, the once-per-process hop notice).

  • src/core/ai/fallback-hop-queue.ts: the process-level queue for the first hop's chat_fallback_hop notice (at most one per process; type-only imports so every drain site loads it cheaply). Drained by stdio MCP dispatch, the CLI op notice channel (src/cli/op-notices.ts) and finishCliTeardown; never by HTTP, because the notice names models.

  • src/core/ai/chat-fallback-planes.ts: where the effective chat_fallback_chain comes from, read without network, subprocess or inference. readChatFallbackPlanes(db) reads env GBRAIN_CHAT_FALLBACK_CHAIN > the config.json key > the DB row (parseDbChatFallbackChain in config-db-merge.ts) with the loader's rule that the first plane that speaks wins even when its value yields no chain, and returns the winner, shadowed values, unreadable planes and the chat_fallback_on_refusal value with its plane. diagnoseChatFallbackEntry checks one entry by presence: provider:model parse, known provider with chat, the provider's own credential (recipe env keys or authPresent; for claude-cli the binary on PATH or GBRAIN_CLAUDE_CLI_BIN), and a price when a key in USER_CHAT_CAP_KEYS sets a user cap. chatFallbackRemovalFix is the per-plane removal guidance (DB gbrain config unset chat_fallback_chain with destructive consent → ask_user; env/file are the user's, with the restart note). Consumers: the chat_fallback_chain doctor check and the behavior_changes notice.

  • src/core/ai/sdk-warnings.ts — AI SDK provider warnings to stderr (#5892). The gateway calls installAiSdkWarningWriter() when it loads, which sets globalThis.AI_SDK_LOG_WARNINGS to a writer with the ai logger signature ({ warnings, provider, model }) that prints each warning in the SDK's own wording to stderr and never the SDK's console.info banner, which landed on stdout and broke --json output (the claude-cli model's v2-compat notice fires on every process's first call). A user-set value (false or a function) is kept. Tests: ai-sdk-warnings.test.ts, json-stdout-ai-warnings.test.ts.

  • src/core/ai/thinking-off.ts: the per-route mapping behind ChatOpts.thinking: 'off' (#5331). thinkingOffControl(modelStr) is the per-model capability table: the provider-options namespace and keys to set, and whether they turn reasoning off (disables) or only set its floor. Full off: native anthropic, deepseek, openrouter with a deepseek/ model (thinking: { type: 'disabled' }), Gemini 2.5 Flash/Flash-Lite (thinkingConfig.thinkingBudget: 0), OpenAI gpt-5.1/5.2/5.4/5.5/5.6 and native particle (reasoningEffort: 'none', sent as reasoning_effort on Chat Completions; pinned by test/ai/recipe-particle.test.ts). Floor: Gemini 2.5 Pro (thinkingBudget: 128), Gemini 3.x (thinkingLevel minimal, or low on 3.7/3.8 Flash and 3.x Pro, where minimal is a 400), gpt-5/-mini/-nano (minimal), o-series (low). Other Gemini 3+ and OpenAI gpt-5/o-series reasoning ids get no option and the headroom. gpt-6 ids have no row (the pinned @ai-sdk/openai drops their effort, and no live check measured them), so they keep the requested cap that fence repair's measured default openai:gpt-6.1-sol is costed against. Only native recipes get an option; the two Google fields are never sent together. applyThinkingOff sets the keys, replacing a configured value under the same key while siblings such as cacheControl or promptCacheKey survive; chat() applies it after the call-scoped providerOptions merge, on the resolved recipe. thinkingOffMaxOutputTokens keeps the requested cap where reasoning turns off and raises a route that keeps reasoning (a floor row, or no row and isThinkingModel) to the thinking headroom. gateway.ts thinkingOffOutputCap(model, requested) wraps it for chat() and for the takes-quality and cross-modal preflight estimates, so they price the cap the call sends. Callers: those two judges and the synthesize triage judge. Rows were checked live on 2026-10-07 (docs/fix-wave-notes/capy-wave-9-followups.md). Pinned by test/ai/chat-thinking-off.test.ts (request bodies per route against a local stub, estimate parity per default panel model) and test/ai/gateway-chat.test.ts.

  • src/core/ai/model-resolver.ts — Recipe-touchpoint validator. assertTouchpoint(recipe, touchpoint, modelId) checks the PROVIDER's capability only (anthropic has no embedding touchpoint; voyage/ollama have no chat) and never gates on the model id. Recipe models: arrays are informational — default-model selection (models[0] for --model <provider> shorthand and env-ready pickers), guard-test fixtures for the repo's own hardcoded defaults, and gbrain providers list display — NOT a runtime allowlist, so frontier models newer than a recipe work without a recipe PR. A nonexistent id surfaces as the provider's own model_not_found at call time; gbrain models doctor live-probes the configured models for a pre-flight check. Exception: gateway.rerank() keeps its own model-list check because each listed reranker id maps to a known request/response wire shape. embeddingDimsForModel matches recipe model_dims keys exactly first, then case-insensitively with BOTH sides folded (configured ids arrive cased, and user-editable recipe tables can carry cased keys; without the fold a cased id would fall through to default_dims and gbrain init would build a wrong-width column), before falling back to default_dims.

  • src/core/ai/openai-latest.ts — latest-model discovery: OpenAI defaults are NEVER pinned. refreshLatestOpenAIModels() (called lazily from the gateway's discoverBeforeOpenAICall, once per reconfigureGatewayWithEngine, just before an OpenAI-routed chat, expansion or embedding call, never on engine connect (#6607: a brain that never calls OpenAI sends nothing even with OPENAI_API_KEY set; when the refresh changes the discovered tiers the gateway re-resolves its models before the call reads them), TTL 24h, 3s-bounded, fail-open, GBRAIN_MODEL_DISCOVERY=off|0 kill switch — the test preload sets it; skipped, even forced, when a gbrain setup receipt for this install recorded provider use as declined — providerDeclineRecord in src/core/setup/local-receipts.ts, an unreadable record fails closed) fetches the account's own GET /v1/models, ranks ids through a conservative grammar (parseOpenAIChatId: bare family aliases gpt-N.M + known tier suffixes sol/pro/terra/luna/nano/mini; dated snapshots, -chat Instant-class, realtime/image and UNKNOWN future suffixes are ignored — rot degrades to newest-known-shape, never a wrong pick), and maps the newest family onto the tier ladder (utility→cheap, reasoning/subagent→mid, deep→top). ONLY ids with a canonical pricing row are eligible (rankOpenAIChatModels's priced filter): BudgetTracker fails closed (no_pricing) under a cost cap, so an unpriced discovered default would brick budget-capped backfills — a newer-unpriced family warns once naming the missing model-pricing.ts row. Result lands in <configDir()>/model-cache.json (atomic tmp+rename); latestOpenAITiers() is the SYNC read (stale cache beats static fallback — TTL gates refresh, never use) consumed by resolveTierDefault's openai entry and the dynamic gpt alias. Pinned by test/openai-latest.serial.test.ts.

  • src/core/ai/openrouter-families.ts — single source of truth for which OpenRouter model families may drive the subagent (tool) loop: OPENROUTER_SUBAGENT_FAMILIES (currently anthropic/ + deepseek/, each backed by a live abort/retry replay pin under test/e2e/) + openrouterModelSupportsSubagentLoop(modelId) (takes the bare OpenRouter id, no openrouter: prefix). Shared by the openrouter recipe's predicate and the subagent handler's auto-route so the two can never disagree; a new family gets its own live pin before joining the list.

  • src/core/ai/provider-env.ts — mergedProviderEnv(cfg, env): THE canonical provider-key/env fold. Maps file-plane config keys (openai/anthropic/openrouter/voyage/dashscope/deepseek/litellm/together/google/azure_openai + azure endpoint/deployment/entra) to the env names recipes read, merges env on top (env wins ONLY for keys carrying a real value — '' and undefined dropped), then applies the GEMINI_API_KEY → GOOGLE_GENERATIVE_AI_API_KEY alias (canonical env name > alias > config fallback). Three consumers: buildGatewayConfig (gateway env), detectCapabilities (capability probe), resolveTierDefault/resolveEffectiveChatModel (key-aware model resolution) — so gateway env and the capability probe cannot drift. #5137: the fold reads the CONFIG_API_KEY_ENV table; providerKeyShadows(fileCfg, env) (pure, names only) lists env values that override a different config-plane key, and providerKeySource names where a recipe's key comes from. Both take the FILE-only config (loadConfigFileOnly), because loadConfig already folds OPENAI/ANTHROPIC/OPENROUTER env values into the config fields. TypeSafe has no config key, so its env alias never shadows. Tests: embedding-key-source.test.ts, embedding-key-source-cli.test.ts.

  • src/core/ai/chat-key-probe.ts — probeChatModelWithKey(model): probeChatModel plus an up-front key check for every keyed provider (recipe auth_env.required against mergedProviderEnv over process env, the gateway env snapshot and the config file; Anthropic keeps probeChatModel's check; keyless recipes pass). The hint names gbrain config set <key> only when CONFIG_API_KEY_ENV has one. Used by the propose_takes and patterns cheap keyless skips (#6705); probeChatModel keeps its lazy A9 contract. Pinned by test/ai/gateway-probe-chat-model.test.ts.

  • src/core/ai/key-warnings.ts — #5137 print seam with one replaceable sink and two once-per-process latches: warnShadowedProviderKeys (called from src/cli.ts for every command except gbrain hook; gbrain serve starts through the same path) names the variable, the config key, that env is in effect, and both fixes (remove the variable from that process's environment and restart it, or gbrain config unset <key>, which also silences it); reportEmbeddingAuthFailure prints embedding_auth_failed (catalogue entry) on the first embedding 401/403, naming the key source in effect and the fix plus gbrain embed --stale. Never prints a key or fragment. Doctor embedding_key_source (src/commands/doctor/checks/embedding-health.ts, engine-free filesystem-lane entry embeddingKeySourceEntry) reports the same shadows and the embedding key's source, and says it sees only the environment doctor runs in.

  • src/core/ai/key-redact.ts — redactProviderKeys(text, env): scrubs every *_API_KEY/*_TOKEN/*_SECRET/*_PASSWORD value in the gateway env, masked echoes (sk-...****abcd) and 8+ character key fragments from upstream error text. The gateway passes it to normalizeAIError (which also scrubs the wrapped error's cause chain in place, keeping status and headers for .cause walkers) on embed, multimodal, expand and chat errors, and applies it to multimodal response bodies.

  • src/core/ai/reranker-readiness.ts — pure leaf answering "is the reranker actually going to run?" for gbrain search modes (buildModesReport.reranker_readiness), doctor's reranker_health, and init's reranker-default write. rerankerReadiness(model, env, { baseUrlOverrides? }) → {model, provider, modelId (alias-canonical), recipeKnown, hasTouchpoint, modelListed, requiredKey (the first MISSING key, else the first required key — this is how Reranker: … VOYAGE_API_KEY present renders; null only for keyless recipes), keyPresent, selfHosted, ready}; the env snapshot comes from the CALLER — the leaf never reads process.env and never imports the gateway (init runs before configureGateway and passes mergedProviderEnv(cfg, process.env) where cfg is loadConfigFileOnly() merged with the brain's DB-plane provider keys via loadConfigWithEngine — file plane alone if that read fails — so a --force re-init sees a Voyage key that lives only in the config table). A recipe with a custom resolveAuth needs no env key (mirrors the gateway); a provider_base_urls override marks the provider self-hosted. describeRerankerFix(r) renders the paste-ready one-liner (unknown-model > missing key + disable command). src/core/ai/reranker-readiness-engine.ts is the engine-plane wrapper doctor and gbrain search modes share: rerankerReadinessForEngine(engine, model) reads the LIVE gateway snapshot (env + base_urls — exactly what rerank() consults, so the verdict cannot disagree with search and a mounted brain's DB plane cannot steer it) and falls back to env > file > DB-plane provider keys + provider_base_urls via loadConfigWithEngine only when no gateway is configured; redactReadinessForRemote (modes-report.ts) strips required_key/key_present/fix from the search_modes op for untrusted callers. test/ai/reranker-readiness.test.ts pins agreement with isAvailable('reranker', m) on an env × model matrix (recipeKnown && hasTouchpoint && keyPresent) so the two predicates cannot drift.

  • src/core/ai/types.ts — provider/recipe types. EmbeddingTouchpoint has optional chars_per_token (default 4, matching OpenAI tiktoken on English) and safety_factor (default 0.8, budget-utilization ceiling), both consulted only when max_batch_tokens is also set; Voyage declares chars_per_token=1 + safety_factor=0.5 to handle dense payloads (CJK/JSON/base64). Pre-split budget = max_batch_tokens × safety_factor / chars_per_token. EmbeddingTouchpoint.multimodal_models?: string[] model-level allow-list for recipes mixing text-only + multimodal models under one touchpoint (Voyage's 12 models share supports_multimodal: true but only voyage-multimodal-3 accepts /multimodalembeddings); when omitted, recipe-level supports_multimodal is sufficient. AIGatewayConfig.embedding_multimodal_model?: string lets embedMultimodal() route to a different model than embedding_model (OpenAI text + Voyage images without flipping the primary pipeline). AIGatewayConfig.embedding_image_ocr_model?: string is its OCR sibling: generateOcrText() routes to it instead of the expansion model; a direct provider:model string, never models.tier-resolved. EmbeddingTouchpoint.trust_custom_dims?: true — passthrough tier for a user-declared --embedding-dimensions on local / bring-your-own-backend recipes (ollama, llama-server, litellm) where the model catalog can't be enumerated; consumed by isCustomDimValidForProvider in src/core/embedding-dim-check.ts AFTER Tier 1 (recipe dims_options) and Tier 2 (provider Matryoshka allowlists), so a recipe that declares fixed options (openrouter) is still governed by those and fixed-dim hosted providers (openai/voyage) stay fail-closed; the provider's /embeddings response-dim validation catches a genuine mismatch pre-storage. Recipe.default_headers?: Record<string, string> (static) and Recipe.resolveDefaultHeaders?(env) (env-templated) seam for per-recipe headers riding alongside auth on every openai-compat touchpoint; mutually exclusive (declaring both throws AIConfigError at gateway-configure time); keys conflicting with the resolved auth header (Authorization, the resolver's custom header) rejected at applyResolveAuth call time so defaults can't shadow auth. Used by OpenRouter for the HTTP-Referer + X-OpenRouter-Title + X-Title attribution triple.