Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript
32.0 KB
Raw Blame

Embedding providers

GBrain ships provider recipes covering Voyage (the new-install default), OpenAI, OpenRouter (single key, many hosted models), the major hosted alternatives, local options, and a universal escape hatch (LiteLLM proxy). Run gbrain providers list to see the live registry; gbrain providers explain --json emits a machine-readable matrix for agents.

This page is the human-readable counterpart: capability per provider, env-var setup, dimensions, cost, and known constraints.

Quick start

gbrain providers list                          # see all providers
gbrain providers env <provider-id>             # see required env vars
gbrain providers test --model openai:text-embedding-3-large   # smoke-test
gbrain init --pglite --model voyage            # use a non-default provider

Init resolves your provider from your keys

gbrain init --pglite auto-detects which provider to use from your provider keys — env vars or the file plane (~/.gbrain/config.json fields like voyage_api_key; env wins when both are set). With VOYAGE_API_KEY set, you get Voyage (voyage:voyage-4 @ 1024d). With OPENAI_API_KEY set, you get OpenAI. Whenever a Voyage key is present — even if a different embedding provider is picked — the mode-bundle reranker default (voyage:rerank-2.5) is already ready, so init writes no reranker row at all (one key covers both; it just prints Reranker: voyage:rerank-2.5 (mode-bundle default)); keyed installs without a Voyage key get search.reranker.enabled false written instead. If multiple provider keys are set, init fires an interactive picker (non-TTY auto-picks the Voyage default when its key is present). With no provider keys at all, init continues keyless (keyword-only search) with a loud notice; recover later with gbrain init --force --embedding-model voyage:voyage-4. Explicit flags (--embedding-model, --no-embedding) always win over key detection.

The resolved provider + dimensions get persisted to ~/.gbrain/config.json atomically, so subsequent runs are deterministic across releases.

TL;DR table

Provider env vars default dims cost ($/1M tokens) local? multimodal?
voyage (default — voyage-4 @ 1024d; rerank-3-lite reranker on the same key) VOYAGE_API_KEY 1024 0.06 (voyage-4) no yes (voyage-multimodal-3)
openai OPENAI_API_KEY 1536 0.13 no no
openrouter OPENROUTER_API_KEY per-model (1536 for the default openai/text-embedding-3-small; unlisted ids require explicit dims) 0.02 no model-dependent
google GOOGLE_GENERATIVE_AI_API_KEY 768 0.025 no no
azure-openai AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_DEPLOYMENT 1536 0.13 no no
minimax MINIMAX_API_KEY 1536 0.07 no no
mistral MISTRAL_API_KEY 1024 0.10 no no
dashscope DASHSCOPE_API_KEY 1024 varies no no
zhipu ZHIPUAI_API_KEY 1024 varies no no
ollama (none — runs locally) 768 0 yes no
llama-server (none — runs locally) user-set 0 yes no
lmstudio (none — runs locally) user-set 0 yes no
litellm LITELLM_API_KEY (optional) user-set varies yes (proxy) yes (backend permitting)
together TOGETHER_API_KEY 768 varies no no
anthropic (no embedding model — chat only) — — — —
deepseek (no embedding model — chat only) — — — —
groq (no embedding model — chat only) — — — —
particle (no embedding model — chat only: particle:particle-1, PARTICLE_API_KEY) — — — —

Note on local providers. Ollama, llama-server and LM Studio have no required API key, so they don't show up in env-detection auto-pick. Pick them explicitly with --embedding-model ollama:<model> to avoid silently routing to a daemon that may not be running. Ollama can also run local chat/expansion models; set those routes explicitly with gbrain config set models.chat ollama:<model> and related per-task model keys.

Existing brains need an explicit migration. Removing an unsupported provider does not convert stored vectors, change their provenance, or authorize paid work. Inspect gbrain migrate embeddings --status, then preview a supported target with --dry-run and obtain approval before running it. See the migration guide.

If first import fails

If gbrain import fails with expected N dimensions, not M, run gbrain doctor. The output will print the exact gbrain config set ... or gbrain migrate embeddings command to repair the mismatch. You should not need to delete ~/.gbrain.

The doctor distinguishes two repair paths:

  • Empty brain (no embedded chunks yet) — drop and re-init at the right dim:

    gbrain init --force --pglite --embedding-model <provider>:<model> --embedding-dimensions <N>
    
  • Non-empty brain — migrate cleanly with the supported migration path (resumable; preview cost with --dry-run first):

    gbrain migrate embeddings --to <provider>:<model> --dim <N>
    

    The explicit-consent playbook is skills/migrations/v0.46.3.0.md.

Decision tree

  • Cost-sensitive, English-only: Ollama (free, local) or Voyage (paid, best quality per dollar).
  • Quality-first: Voyage voyage-4-large (1024-2048 dims, ~3-4× more dense tokens than OpenAI tiktoken).
  • Code-heavy brain (gstack per-worktree, source repos): Voyage voyage-code-3 (1024 default; supports 256/512/1024/2048), or voyage-code-4 (hosted, flexible dims, $0.12/M). Tuned on programming languages. Voyage publishes head-to-head numbers showing it outperforms their general flagships on code retrieval (voyageai.com/blog). For gstack's per-worktree pglite-backed code brain, this is the right default — see Topology 3 in docs/architecture/topologies.md.
  • Reranking pair: Voyage rerank-3-lite ($0.02/M) is the mode-bundle default and rides the same VOYAGE_API_KEY as embeddings; rerank-3 ($0.05/M) and the older rerank-2.5 / rerank-2.5-lite are also allowlisted. A brain that set search.reranker.model keeps its own model; gbrain config set search.reranker.model voyage:rerank-2.5 restores the previous default. Without the key search fails open in fusion order and gbrain search modes says so.
  • Local reranking (no API spend): llama-server-reranker recipe — point gbrain at your own llama-server --reranking instance running Qwen3-Reranker. Same gateway.rerank() seam, $0 per call. Walkthrough in docs/ai-providers/llama-server-reranker.md.
  • One key for many hosted models: OpenRouter. Set OPENROUTER_API_KEY and use openrouter:<provider>/<model> for chat against GPT-5.2, Claude 4.x, Gemini 3, DeepSeek, and dozens more without juggling per-provider keys. Embedding catalog includes OpenAI, Google, Qwen, BGE-M3.
  • Enterprise compliance: Azure OpenAI (data residency + private endpoints) or self-hosted via llama-server / Ollama.
  • China region: DashScope (Alibaba) or Zhipu (BigModel). DashScope's international endpoint at dashscope-intl.aliyuncs.com; override provider_base_urls.dashscope for the China endpoint.
  • OSS local, full control: llama-server (llama.cpp) for any GGUF model; Ollama for the curated catalog; LM Studio when you'd rather load the model from a GUI than a command line.
  • Anything else: LiteLLM proxy. Run LiteLLM in front of any provider (Bedrock, Vertex, Cohere, Jina, Fireworks, etc.) and point gbrain at it via LITELLM_BASE_URL.

Per-provider details

OpenAI

The main alternative to the Voyage default (its flexible-dim text-embedding-3 models can keep an existing column width during a provider migration). Set OPENAI_API_KEY. Models: text-embedding-3-large (3072 max, 1536 default), text-embedding-3-small (1536). Matryoshka via the dimensions field — gbrain pins it from embedding_dimensions config so existing 1536-dim brains stay aligned across SDK upgrades.

Optional OPENAI_BASE_URL — point the native OpenAI provider at an OpenAI-compatible gateway. A bare host is normalized to carry the /v1 suffix automatically (so https://gw.example.com and https://gw.example.com/v1 both work); when unset, the SDK's default endpoint is untouched. With OPENAI_BASE_URL set, embedding does not require OPENAI_API_KEY — the override targets local OpenAI-compatible servers (LM Studio, vLLM) that ignore auth, so gbrain sends a placeholder key instead of refusing. ANTHROPIC_BASE_URL gets the same normalization for Anthropic chat/expansion calls; GOOGLE_GENERATIVE_AI_BASE_URL does the same for Gemini with /v1beta (Google Gemini).

Voyage AI

The default provider — new installs get voyage-4 @ 1024d ($0.06/M) plus the rerank-3-lite reranker on the same key. Best-in-class quality on the Voyage 4 family (Jan 2026 release). Set VOYAGE_API_KEY. Models: voyage-4-large, voyage-4, voyage-4-lite, voyage-4-nano, voyage-code-4 (code-tuned, hosted, flexible dims, $0.12/M), voyage-3.5, voyage-code-3, voyage-finance-2, voyage-law-2, voyage-multimodal-3 (text + image).

Voyage 4 family shares an embedding space across all variants, so you can index with voyage-4 and later point the query model at voyage-4-large or voyage-4-lite without reindexing. Dims: 256, 512, 1024, 2048. 2048 exceeds pgvector's HNSW cap of 2000 — those brains fall back to exact vector scans (still correct, just slower).

Routed through the Vercel AI Gateway (provider_base_urls.voyage = https://ai-gateway.vercel.sh/v1), the embedding request also carries the width as providerOptions.voyage.outputDimension, which the gateway reads instead of Voyage's root output_dimension; that field is sent only when the hostname is exactly ai-gateway.vercel.sh, because Voyage's own API rejects it with HTTP 400. If a proxy returns a different width than embedding_dimensions, the embed fails before anything is written and the error names the proxy as the likely cause; fix the proxy's width parameter before considering a migration.

Voyage also serves four hosted rerankers at POST /v1/rerank — its current rerank-3 ($0.05/M) and rerank-3-lite ($0.02/M), plus the older rerank-2.5 ($0.05/M) and rerank-2.5-lite ($0.02/M) (prices verified 2026-10-14). All four share one request/response wire, so switching is a config change and nothing else.

rerank-3-lite is the mode-bundle reranker default, so no config row is needed for it (an explicit search.reranker.model equal to it is exactly what gbrain doctor's search_mode check calls redundant). It replaced rerank-2.5 as the default after gbrain-evals #148 (v0.10.93) measured 60% lower rerank cost at 50 candidates × 1,400 tokens with quality within noise. The new default reaches only brains that never set search.reranker.model; a brain with an explicit model keeps it.

To go back to the previous default, or to move to rerank-3, set the model explicitly:

gbrain config set search.reranker.model voyage:rerank-2.5   # roll back to the previous default
gbrain models doctor          # green `reranker_config voyage:rerank-2.5` is the whole test

Say to your agent: "switch my reranker back to voyage rerank-2.5" — "check my model config" (your agent runs gbrain config set search.reranker.model voyage:rerank-2.5 then gbrain models doctor).

For brains that index source code (gstack's per-worktree pglite-backed code brain — see Topology 3 in docs/architecture/topologies.md), prefer voyage-code-3 over voyage-4-large. Voyage tunes it on programming languages and publishes head-to-head numbers vs their general flagships on code retrieval. Configure at install time:

gbrain init --pglite --embedding-model voyage:voyage-code-3 --embedding-dimensions 1024

To switch an existing brain, run gbrain migrate embeddings --to voyage:voyage-code-3 --dim 1024 (works on both engines; resumable, cost-previewed with --dry-run — see docs/guides/embedding-migration.md). gbrain config set embedding_model is refused — the schema column has to resize, and the migration command is the path that does that safely.

gbrain reindex-code will print a recommendation when run against a brain whose configured embedding model isn't code-tuned; suppress with GBRAIN_NO_CODE_MODEL_NUDGE=1 if you've intentionally chosen another model (single-vendor procurement, compliance, etc.).

Google Gemini

Set GOOGLE_GENERATIVE_AI_API_KEY (the AI Studio public API key). Models: gemini-embedding-2 (current, GA 2026-04-22) and gemini-embedding-001. Default 768 dims; Matryoshka up to 3072 (--embedding-dimensions 1536 / 3072). Cheap. The two models' vector spaces are not interchangeable — to move an existing brain from -001 to -2, use gbrain migrate embeddings --to google:gemini-embedding-2 --dim <N>, not config set. Optional GOOGLE_GENERATIVE_AI_BASE_URL points native Gemini chat, expansion and embedding requests at a proxy or gateway; a URL without a version segment gets /v1beta. See Google Gemini.

GCP service-account / Vertex AI auth (Vertex ADC) is not supported; only the AI Studio API key path is wired.

gemini-embedding-001 takes at most 2,048 tokens per input and the endpoint truncates longer inputs silently (autoTruncate defaults on), which embeds a lossy vector rather than failing. gbrain declares the cap so chunks on -001 are capped at 1,228 estimated tokens and split instead (see Per-input token caps). gemini-embedding-2's per-input limit is not declared yet.

OpenRouter

Single OpenAI-compatible API for fan-out to OpenAI, Anthropic, Google, DeepSeek, Meta Llama, Qwen, and dozens of other hosted providers. One key, many models. Set OPENROUTER_API_KEY or openrouter_api_key in ~/.gbrain/config.json, then use openrouter:<provider>/<model> (e.g. openrouter:openai/gpt-5.2, openrouter:anthropic/claude-sonnet-4.6).

Embedding: openai/text-embedding-3-small (1536d default, Matryoshka shrink to 512/768/1024). The recipe carries verified per-model native dims for its catalog — openai/text-embedding-3-large (3072), qwen/qwen3-embedding-8b (4096), bge-m3 (1024) — so opting in via --embedding-model openrouter:<id> plans the right column width automatically. Any id NOT in that list (including google/gemini-embedding-2-preview, whose width is unverified) has no silent default: you must pass explicit dimensions (--embedding-dimensions <N> or embedding_dimensions config) or the command errors with the fix. Pricing matches the upstream provider (OR adds a small markup).

Chat: every chat model OR proxies works through /v1/chat/completions. The recipe lists 8 curated entry points (GPT-5.2 family, Claude 4.5/4.6/4.7, Gemini 3 Flash Preview, DeepSeek); any other OR catalog ID also works. Tool-calling envelope is supported by the OR endpoint, but per-model capability varies — check https://openrouter.ai/models before counting on tools for a specific slug.

Optional env:

  • OPENROUTER_BASE_URL — point at a self-hosted OR-compatible proxy.
  • OPENROUTER_REFERER (default https://gbrain.ai) and OPENROUTER_TITLE (default gbrain) — attribution headers for OR's leaderboard. Forks running gbrain inside a different agent stack (OpenClaw deployments etc.) should set these so their traffic gets attributed to them, not gbrain.

Subagent loops: gbrain's subagent infrastructure hard-pins to Anthropic-direct (stable tool_use_id across crashes/replays). OR-routed Anthropic is rejected at submit time regardless of the recipe flag. If you want the price/availability story OR offers for tool-calling, use it for chat only and keep an Anthropic key for subagent work.

Azure OpenAI

Enterprise OpenAI behind Azure tenancy. Required env: AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT (e.g. https://my-resource.openai.azure.com), AZURE_OPENAI_DEPLOYMENT (the deployment name from your Azure portal). Optional: AZURE_OPENAI_API_VERSION (defaults to 2024-10-21).

Unlike vanilla OpenAI, Azure uses api-key: header (not Authorization: Bearer) and a templated URL with ?api-version= query param — gbrain handles both via the recipe's resolveAuth + resolveOpenAICompatConfig overrides.

Models: text-embedding-3-large, text-embedding-3-small, text-embedding-ada-002 (your Azure deployment must serve the requested model).

Mistral AI

Set MISTRAL_API_KEY. Model: mistral-embed (fixed 1024 dims; the API rejects any dimension parameter, so leave --embedding-dimensions at 1024). Chat and expansion run on the same key (mistral:mistral-large-latest, mistral:mistral-small-latest; unpriced in gbrain's tables, so a user cost cap needs gbrain pricing set), so a brain can keep embed + expansion + chat on one provider.

Region: api.mistral.ai (the default) is EU-hosted by default, but Mistral's terms allow a request to be served outside the EU unless your organization disables transfers; the default endpoint carries no regional-inference assurance. Inference pinned to the EU is Mistral's opt-in regional endpoint, billed at 1.1x list price: gbrain config set provider_base_urls.mistral https://api.eu.mistral.ai/v1, then verify with gbrain config get provider_base_urls.mistral. gbrain does not switch to it for you, and ships no separate mistral-eu recipe.

MiniMax (海螺AI)

Set MINIMAX_API_KEY. Optional MINIMAX_GROUP_ID for org-scoped accounts. Model: embo-01 (1536 dims).

MiniMax's API takes a type: 'db' | 'query' field for asymmetric retrieval. gbrain sends everything as type='db' (symmetric retrieval, same vector space for indexing and queries); asymmetric query encoding is not implemented for this recipe.

DashScope (Alibaba)

Set DASHSCOPE_API_KEY. International endpoint at dashscope-intl.aliyuncs.com by default; override provider_base_urls.dashscope for the China endpoint. Models: text-embedding-v3 (current; Matryoshka 64-1024 dims), text-embedding-v2, text-embedding-v1.

CJK-dominant content tokenizes denser than OpenAI tiktoken; gbrain declares chars_per_token: 2 so the batch pre-split leaves headroom.

Per-input caps are declared (see Per-input token caps): text-embedding-v1 / v2 take at most 2,048 tokens per input (one over-length item fails the whole request with a 400), v3 takes 8,192. New chunks on v1/v2 are capped at 1,228 estimated tokens instead of 2,000; a brain chunked before this declaration is split on its next gbrain embed / embed --stale pass and the split pieces re-embed.

Zhipu AI (BigModel)

Set ZHIPUAI_API_KEY. Models: embedding-3 (current; Matryoshka 256-2048 dims), embedding-2. The default is 1024 (HNSW-compatible). The 2048-dim option works but falls into the exact-scan branch (see Voyage 4 Large note above).

Per-input caps are declared: embedding-3 takes at most 3,072 tokens per string (chunks capped at 1,843 estimated tokens), embedding-2 512 (chunks capped at 307). See Per-input token caps.

Ollama (local)

No env required — Ollama runs unauthenticated locally. Optional OLLAMA_BASE_URL (default http://localhost:11434/v1) and OLLAMA_API_KEY (for auth-enabled deployments).

Recipe ships with nomic-embed-text (768d, recommended), mxbai-embed-large (1024d), all-minilm (384d), plus the larger modern embedders qwen3-embedding:8b (4096d) and snowflake-arctic-embed2 (1024d) — those are the pullable Ollama library tags. The spellings qwen3-embed-8b / snowflake-arctic-embed-l-v2 are not pullable tags but are still accepted so brains initialized with them keep validating. gbrain providers test --model ollama:nomic-embed-text smoke-tests the local install.

The recipe default is nomic-embed-text's 768 dims. If you run one of the larger models, declare its native dimension with --embedding-dimensions <N> at init — gbrain trusts the value you declare for local recipes instead of rejecting a non-768 width.

Ollama also exposes an OpenAI-compatible /v1/chat/completions endpoint, so gbrain can use local chat models for query expansion, think, facts extraction, and dream synthesis. A minimal local setup:

ollama pull nomic-embed-text
ollama pull qwen2.5-coder:14b

gbrain init --pglite --embedding-model ollama:nomic-embed-text --embedding-dimensions 768
gbrain config set models.chat ollama:qwen2.5-coder:14b
gbrain config set models.expansion ollama:qwen2.5-coder:14b
gbrain config set models.think ollama:qwen2.5-coder:14b
gbrain config set models.dream.synthesize ollama:qwen2.5-coder:14b
gbrain config set models.dream.synthesize_verdict ollama:qwen2.5-coder:14b
gbrain config set models.dream.patterns ollama:qwen2.5-coder:14b
gbrain config set facts.extraction_model ollama:qwen2.5-coder:14b

The recipe marks Ollama chat as not tool-capable. That is enough for local reasoning flows, but not for gbrain's subagent/tool loop; keep models.tier.subagent on a provider with tool calling when you need that path.

llama-server (local, llama.cpp)

llama.cpp's llama-server --embeddings endpoint. No env required. Optional LLAMA_SERVER_BASE_URL (default http://localhost:8080/v1) and LLAMA_SERVER_API_KEY.

User-driven models: launch llama-server with --model <gguf-path> --embeddings, then run gbrain init --embedding-model llama-server:<your-id> --embedding-dimensions <N>. gbrain trusts the dimension you declare (you know the GGUF you launched); the recipe refuses the implicit shorthand --model llama-server because there's no canonical first model.

LM Studio (local)

LM Studio's OpenAI-compatible local server, on default port 1234 (distinct from Ollama's 11434 and llama-server's 8080). No env required — the local server ignores auth. Optional LMSTUDIO_BASE_URL (default http://localhost:1234/v1) and LMSTUDIO_API_KEY.

User-driven models, like llama-server: load an embedding model in the app and start the local server, then declare both the model id and its native width. The recipe ships no catalog and no default dimension, so --embedding-dimensions <N> is required — and trusted as declared, because only you know which model you loaded.

gbrain init --pglite --embedding-model lmstudio:<id-from-the-app> --embedding-dimensions <N>

The app's own /v1/models listing is the source for <id-from-the-app>, and gbrain doctor is the check afterwards. Note that gbrain providers test takes its probe width from the recipe's default dimension, so — as with llama-server and the LiteLLM proxy — it reports "not configured or not ready" here rather than smoke-testing the server.

Say to your agent: "Set up gbrain with LM Studio as my embedding provider" — "Initialize brain against my local LM Studio server" — "Is my brain set up right?" Tell it the model id you loaded and the width that model emits; it runs the init above and reports what it picked.

Two ways to reach LM Studio. The native openai: provider with OPENAI_BASE_URL set also reaches it, keylessly (see the OpenAI section above) — the shortest path when you are already configured for OpenAI and only want the embeddings served locally. Prefer the lmstudio: recipe when you want LM Studio as its own provider: OPENAI_BASE_URL redirects the whole native-OpenAI path (embedding, chat and expansion), while lmstudio: carries its own LMSTUDIO_BASE_URL and leaves openai: pointed at OpenAI. The recipe also prices local embedding at $0, so a --max-cost-bounded gbrain embed / gbrain reindex budgets it like Ollama and llama-server instead of refusing a model it cannot price.

If search stops returning semantic hits, reload the model. A loaded LM Studio instance can drift into a state where embedding latency rises by orders of magnitude while the endpoint still answers — so it passes the reachability probe but misses the query-embed deadline (GBRAIN_QUERY_EMBED_TIMEOUT_MS, default 6s) on every query. The query still returns, keyword-only, stamped embed_timeout in the response's degraded[] with a once-per-process warning on stderr; recall drops until you reload the model in the app. Raise the timeout on slower hardware.

LiteLLM proxy (universal escape hatch)

Run LiteLLM in front of any provider — Bedrock, Vertex, Cohere, Jina, Fireworks, OctoAI, etc. The proxy normalizes everything to the OpenAI-compatible API; gbrain points at the proxy via LITELLM_BASE_URL and proxies the call.

This is the catch-all for "my provider isn't in the list above." Set up LiteLLM, then gbrain init --embedding-model litellm:<your-model-id> --embedding-dimensions <N>.

Include the /v1 suffix in LITELLM_BASE_URL if your proxy serves the OpenAI route there (e.g. http://localhost:4000/v1). Many LiteLLM deployments expose the OpenAI-compatible API only under /v1; pointing gbrain at the bare host 404s or fails authentication with no hint. gbrain trusts the dimension you declare for the proxy-backed model — the proxy's backend, not gbrain, decides the true width — so --embedding-dimensions <N> is required and accepted as-is.

Choosing dimensions

Three numbers matter:

  1. Provider's native dims: each model has a "true" output dim (e.g. OpenAI text-embedding-3-large is 3072 native).
  2. Matryoshka reductions: most modern providers let you request a smaller vector via the dimensions field.
  3. HNSW cap: pgvector's HNSW index supports up to 2000 dims. Brains above that fall back to exact vector scans (slower but correct; gbrain handles the SQL automatically via chunkEmbeddingIndexSql in src/core/vector-index.ts).

For most users: stay at 1024 or 1536. Bigger isn't better below the noise floor; smaller saves disk + RAM with marginal recall loss on Matryoshka providers.

Per-input token caps

Some hosted encoders reject (or silently truncate) any SINGLE input over a per-input token limit, far below their batch budget. A recipe declares the documented limit per model (max_input_tokens), and the chunker caps new chunks at floor(limit × 0.6) estimated tokens, never above the 2,000-token default, splitting rather than truncating; gbrain embed and embed --stale split already-stored chunks the same way before sending them. Declared today: NVIDIA nvidia/nv-embedqa-e5-v5 512; Zhipu embedding-2 512, embedding-3 3,072; DashScope text-embedding-v1 / v2 2,048, v3 8,192; Google gemini-embedding-001 2,048; OpenAI text-embedding-3-* 8,191; Mistral mistral-embed 8,192 (the last three are above the default chunk cap, so chunking is unchanged). GBRAIN_MAX_CHUNK_TOKENS overrides the resolved cap for a model no recipe declares.

When a model declares no cap and the provider answers a single input with an over-length 400 (input length exceeds the context length, exceeds maximum allowed token size N, Range of input length should be [1, N], …), the embed pass derives a smaller cap from the failure (the limit the message names × 0.6, otherwise half the largest failed chunk's estimate), keeps it for that model for the rest of the process, and splits the failing page at once; the pieces embed on the next pass, and later pages are split before their first request. The descent is finite: three attempts per page per process and never below 128 tokens, after which the failure is reported with the fix (GBRAIN_MAX_CHUNK_TOKENS=<limit>, a recipe max_input_tokens entry, or embed_skip on the page). A model whose cap IS declared gets no derived cap: the declaration is what to correct. Unrelated 400s (bad key, unknown model) and batch-level token errors never rewrite a chunk.

Bulk-embed failure knobs

Two env vars tune how bulk embeds (gbrain embed --stale and the autopilot/minion cycles that ride the same stale path) behave under sustained failures. Both matter most against local serial servers (Ollama at -np 1, llama-server), where re-sending doomed requests compounds into congestion collapse:

Env var Default What it does
GBRAIN_EMBED_QUARANTINE_AFTER 3 Consecutive zero-progress attempts (no chunk embedded) before a page is quarantined for the rest of the process. An attempt that embeds ANY chunk resets the page's counter — partial progress shrinks the stale set, so the next pass sends a smaller request, not the identical doomed one. Quarantine is keyed per page (source_id::slug) and announced once on stderr; later passes skip quarantined pages with a count. Process-lifetime by design: restarting retries deliberately, and frontmatter.embed_skip is the permanent block. Non-numeric or non-positive values fall back to 3.
GBRAIN_EMBED_MAX_BATCH_TOKENS (unset) Token cap per embedding request for recipes that ship without one. ollama, lmstudio, and litellm declare no_batch_cap because real capacity depends on the operator's server; llama-server declares no token cap either, so the env cap applies to it too. When set, chunk texts are pre-split into sub-batches within cap × safety_factor (default 0.8) tokens, estimated via the recipe's chars_per_token (default 4). A recipe-declared cap always wins; invalid values are ignored. Read once from the environment at process start, never at call time. Without it, no_batch_cap recipes still get a conservative 16-item sub-batch cap so the per-call embed timeout bounds a fixed amount of work. llama-server is not a no_batch_cap recipe: it ships max_batch_items: 32 (its launch --batch-size default), so it gets that item cap rather than the 16-item fallback. The chars_per_token estimate is tuned for English; CJK text usually tokenizes far denser (often close to one token per character), so the default estimate can undercount several-fold. Set the cap with headroom against your model's real tokenizer. The pre-splitter never splits an individual text, so one oversized input is still sent whole; llama-server returns an error for a single embedding input larger than its physical batch (-ub, see llama.cpp #7389). Raise -ub or lower your chunk size if long chunks fail.

My provider isn't listed

Four options:

  1. Use OpenRouter when the provider/model is available through OR's OpenAI-compatible API (covers most hosted chat models + a growing embedding catalog).
  2. Use LiteLLM proxy (above) — the universal escape hatch. Works for 100+ providers.
  3. Open a feature request at github.com/garrytan/gbrain/issues with the provider's API docs URL and a setup snippet. Recipes are ~30-40 lines of TypeScript.
  4. Submit a recipe: clone, copy src/core/ai/recipes/voyage.ts as the gold-standard openai-compat template, register in src/core/ai/recipes/index.ts, add a per-recipe smoke test under test/ai/recipe-<name>.test.ts. The IRON RULE regression test (test/ai/recipes-existing-regression.test.ts) pins the shared auth contract for the existing recipes.

Switching providers on an existing brain

Embedding dimensions are baked into the schema at gbrain init time. gbrain config set embedding_model and gbrain config set embedding_dimensions are refused — the schema column has to resize alongside the config, and config set only touches the config row.

Use gbrain migrate embeddings on both PGLite and Postgres. It coordinates schema, config, vector invalidation and the re-embed pass, with a resumable checkpoint. Start with --status, choose a supported model and width, then preview the cost before approving any write:

gbrain migrate embeddings --to voyage:voyage-4 --dim 1024 --dry-run

Only after the user approves the target, affected data and cost should the agent run the same command with --yes instead of --dry-run. Keep a verified full backup and quiesce embedding writers first. Do not use a config edit or a wipe-and-reinitialize shortcut to switch an existing brain. See the migration guide.

gbrain doctor 8c "alternative_providers" surfaces unconfigured providers whose env is already set — useful when you've configured OpenAI but also have e.g. VOYAGE_API_KEY exported and want to know you can switch without extra setup.