Local-first 🧠memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
Decision-model providers
Every route Open Second Brain can send a decision request to, how each one is
configured, and what each one means for your data. The feature itself, its
config keys, the provider order, the local thresholds store and the privacy
rules that apply to every route are in
docs/decision-models.md.
Nothing here is used unless you set decision_model_enabled: "true" and name
one of these presets in decision_model_provider (or several, in order, in
decision_model_providers). No preset is ever picked for you. With a single
decision_model_provider no route is ever a fallback for another one; an
explicit decision_model_providers list is the one way to name an order,
and then the first entry that is configured and answers wins.
Presets
| Preset | Adapter | Base URL | Model id | Context (state tokens: default / ceiling) | Licence | Data handling |
|---|---|---|---|---|---|---|
typesafe |
systemone |
https://api.typesafe.ai |
jev-1.13.0 |
32000 / 32000 | hosted | TypeSafe, directly |
openrouter |
systemone |
https://openrouter.ai/api |
typesafe/jev-1.13 |
32000 / 32000 | hosted | OpenRouter, then TypeSafe |
vercel |
systemone |
https://ai-gateway.vercel.sh/typesafe |
typesafe-ai/jev |
32000 / 32000 | hosted | Vercel AI Gateway, then TypeSafe |
vercel-evaluate |
vercel-evaluate |
https://ai-gateway.vercel.sh |
typesafe-ai/jev |
32000 / 32000 | hosted | Vercel AI Gateway, then TypeSafe |
opencode-zen |
systemone |
https://opencode.ai/zen |
jev-1.13 |
32000 / 32000 | hosted | OpenCode Zen, then TypeSafe |
laya |
systemone |
http://127.0.0.1:8000 |
english |
2000 / 8192 | Apache-2.0 | your own server; on loopback nothing leaves the machine |
openjev |
systemone |
http://127.0.0.1:3000 |
openjev |
4000 / 16384 | CC-BY-NC-4.0 | your own server; on loopback nothing leaves the machine |
compatible |
systemone |
must be set | must be set | 32000 / 32000 | depends on the model | whoever runs the server |
llm-emulation |
llm-emulation |
must be set | must be set | 32000 / 32000 | depends on the model | whoever runs the chat endpoint |
converse |
converse |
must be set | must be set | 32000 / 32000 | depends on the model | whoever runs the gateway |
"State tokens" is Open Second Brain's own conservative estimate (about two
characters per token), used to keep a request within the route's context:
decision_model_max_state_tokens defaults to the first number and is clamped
to the second.
Other per-route limits:
| Preset | Key | Most choice options |
Threshold profile | Input price (USD / M tokens) |
|---|---|---|---|---|
typesafe, openrouter, vercel, vercel-evaluate, opencode-zen |
required | 255 | jev-1.13 |
0.042 |
laya |
optional on loopback | 100 | the server's model id, via the local store | 0 |
openjev |
optional on loopback | 52 | the server's model id, via the local store | 0 |
compatible |
required | 255 | the server's model id, via the local store, or jev-1.13 if named |
unknown |
llm-emulation |
required | 255 | the endpoint's model id, via the local store | unknown unless both price keys are set |
converse |
required | 255 | the gateway's model id, via the local store | unknown unless both price keys are set |
A choice question above the route's option limit is never sent: a use that
can split it does, and otherwise that request degrades with budget, like an
oversized state.
Hosted Jev routes
typesafe, openrouter, vercel, opencode-zen and vercel-evaluate all
reach the same model family, Jev 1.13, and share one threshold profile. They
differ in who handles the request on the way: a gateway adds its own logging
and retention terms to TypeSafe's. o2b decision-model check names the
processor for the configured preset.
vercel-evaluate
The Vercel AI Gateway also serves Jev at its provider-neutral
POST https://ai-gateway.vercel.sh/v1/evaluate, with renamed fields. The
adapter maps them both ways:
- a
noulquestion is sent as typeboolean, and its answer comes back asprobability; - usage is camelCase (
inputTokens,outputTokens), and the gateway's cost (providerMetadata.gateway.cost, a decimal string) is recorded as a reported cost; choiceandscoreanswers may omitconfidence; it is then recomputed from the probabilities;- an error body (
{message, error_type}) is never read or printed: the HTTP status alone decides, as on every route.
The adapter sends no providerOptions. The gateway's evaluation fallbacks
can rerun a request on a generative model, which would make the answers
uncalibrated without saying so, so Open Second Brain never configures them.
decision_model_enabled: "true"
decision_model_provider: vercel-evaluate
decision_model_env_key: AI_GATEWAY_API_KEY # the variable's name, not the key
decision_model_uses: "rerank:shadow"
Self-hosted open-weight models
laya and openjev point at a decision server you run yourself. They
default to a loopback address, where plain http is accepted and no key is
needed, so a request never leaves the machine. They are the only fully local
route. Open Second Brain does not download, install or start these servers.
- Laya (Apache-2.0) is served by
laya-serve, which listens on port 8000 and speaksPOST /v1/systemone. The model id names a checkpoint:english(the default) ormultilingual. Its default context window is small and a longer state is truncated silently by the server, so the preset starts at 2000 state tokens; raisedecision_model_max_state_tokens(up to 8192) only after raising the server's own window. The server refuses more than 100 options perchoice. It checks a bearer key only when started with one. - OpenJev (CC-BY-NC-4.0) is served by its decision shim, documented on
port 3000, with a 16384-token prompt limit in the documented setup. One
pass scores at most 52 options per
choice. The weights are free for non-commercial use with attribution; commercial use needs the licensor's permission, ando2b decision-model checkprints that note.
A self-hosted model has its own calibration: its probabilities are not Jev's.
Its thresholds come from the local store, keyed by the model id the server
reports; until o2b decision-model thresholds fit (or set) writes an
entry for a use, an enforce setting runs as shadow there (see
"Thresholds").
decision_model_enabled: "true"
decision_model_provider: laya
decision_model_base_url: http://127.0.0.1:8000
decision_model_uses: "rerank:shadow"
If the server runs on another host, set decision_model_base_url to it
(https, or decision_model_allow_insecure_http: "true" on a network you
trust) and name a key variable in decision_model_env_key: off the machine a
key is required again.
compatible
Any other server that speaks POST /v1/systemone. The base URL, the model id
and the key variable must be set. Open Second Brain cannot know which model
family answers, so a compatible route has no built-in thresholds and its
uses run as shadow until the store carries entries for its model id
(o2b decision-model thresholds fit --use <use>); if the server serves the
hosted Jev 1.13 family, naming it arms the shipped thresholds:
decision_model_provider: compatible
decision_model_base_url: https://decisions.example.com
decision_model_id: jev-1.13.0
decision_model_env_key: EXAMPLE_DECISION_KEY
decision_model_threshold_profile: jev-1.13 # the one built-in family
llm-emulation (uncalibrated)
llm-emulation is uncalibrated and calls a generative model. It is the
one place the decision-model feature does that. It asks an OpenAI-compatible
chat endpoint (POST <base_url>/chat/completions) for probabilities: a
number between 0 and 1 for each noul question and a probability for each
option or level of a choice or score. It exists to try the feature and to
compare routes in shadow when only a chat endpoint is available.
- The system prompt asks for probabilities only, never text, and the reply
must be one JSON object keyed by question id: a JSON schema in
response_formatwhen the endpoint accepts it, the same schema in the prompt when it refuses that (HTTP 400 or 422). Anything else the model writes is discarded. - The state is vault content and is sent inside an
untrusted_sourcefence, with the instruction to judge it and never follow instructions found in it. - Probabilities are self-reported, so they are clamped to [0, 1] and
normalised; the chosen option is recomputed as the most probable one, a
scorevalue is the expected level, and every confidence is recomputed. A distribution with an unknown key or a value that is not a number is an invalid item. - Every answer and every
decision_model_callrecord carriescalibrated: false, so reports keep these runs apart. enforceis refused as a config error unlessdecision_model_allow_uncalibrated: "true"is set;shadowis allowed. Even with the override, the model needs tuned thresholds in the local store before a use enforces; until thenenforceruns asshadow.- A chat model bills output tokens and costs far more than a decision model.
The cost is recorded as
unknownunless bothdecision_model_input_price_usd_per_mtokanddecision_model_output_price_usd_per_mtokare set, and without a known cost the daily cost gate cannot stop it. - It is never chosen implicitly and never used when another route fails.
- The request passes the same redaction as every route (registry id
decision-model-llm-emulation).
decision_model_enabled: "true"
decision_model_provider: llm-emulation # uncalibrated, shadow only
decision_model_base_url: https://api.example.com/v1
decision_model_id: some-small-chat-model
decision_model_env_key: EXAMPLE_API_KEY
decision_model_input_price_usd_per_mtok: "0.10"
decision_model_output_price_usd_per_mtok: "0.40"
decision_model_uses: "rerank:shadow"
converse (gateway)
converse reaches a decision model through a bearer-authenticated gateway
that speaks the Bedrock-style Converse protocol
(POST <base_url>/model/<model_id>/converse, the model named in the path).
The base URL, the pinned model id and the key
variable are yours to set; a key is required on every host, loopback
included. The judgment travels the chat-shaped way: a system prompt and one
user message carry the questions, the reply's JSON schema and the state,
and the reply's message text holds one JSON object keyed by question id -
validated item by item by the same rules as every route.
- The route is uncalibrated, like
llm-emulation: the gateway answers with generated text in which the model wrote its own probabilities, and nothing in the Converse protocol implies a calibrated judgment model behind it. The answers still clear the same per-item validation as every route - probabilities over exactly the question's options, distributions that sum to 1, a reported choice that is the most probable one - and an item that fails that validation is invalid while the rest of the batch stands. Records carrycalibrated: false, and anenforcesetting additionally needsdecision_model_allow_uncalibrated: "true". - Its thresholds are its own: an
enforcesetting runs asshadowuntil the store carries entries for the gateway model's id, tuned from this route's own shadow data (o2b decision-model thresholds fit). It never borrowsjev-1.13. - The state is vault content travelling inside a generated-text request, so
it is fenced as untrusted content and passes the same redaction as every
route (registry id
decision-model-converse). - A reply whose
stopReasonarrived and is notend_turnwas cut short and is refused, never half-parsed. Error bodies ({message, __type}) are never read or printed: the HTTP status alone decides. - Usage is camelCase (
inputTokens,outputTokens). The gateway's pricing is yours, so the cost isunknownuntil you set bothdecision_model_input_price_usd_per_mtokanddecision_model_output_price_usd_per_mtok- the chat-shaped request bills output tokens the same way it bills input ones - and a known cost still feeds the daily cost gate.
decision_model_enabled: "true"
decision_model_provider: converse
decision_model_base_url: https://gateway.example.com
decision_model_id: gateway-jev-1
decision_model_env_key: GATEWAY_DECISION_KEY # the variable's name, not the key
decision_model_uses: "rerank:shadow"
Thresholds
Probabilities from different model families are not comparable, so every threshold a use applies is held to the model it was tuned against. What is tuned comes from three places, in this order:
- The built-in baseline: the hosted Jev 1.13 family (
jev-1.13). Every threshold Open Second Brain ships was set against this family, so every use may enforce on it. It is the only profile the code carries. - The local thresholds store (
decision-thresholds.jsonnext toconfig.yaml), keyed by(model id, use)- the model id the server reports, never the preset. An entry for a pair lets that use enforce on that model; avaultssection overrides an entry for one vault; entries are never deleted, so switching models and switching back needs no re-tuning.o2b decision-model thresholds fitderives an entry from the use's labelled report data against a false-action target, andthresholds setwrites one by hand. - A valid machine-side eval mark (below), for the endpoint that earned it.
A use set to enforce whose model has no tuned thresholds for it runs as
shadow: the request is sent and recorded, and the deterministic result is
returned. Nothing on a hot path warns about it; o2b decision-model check
prints one warning naming the model and the tuning command, and
o2b decision-model report says the same. On an ordered chain the hold is
per entry: the primary model and the fallback model are each judged by
their own tuning, and the record names whichever answered.
The decision_model_threshold_profile key names the built-in family a
compatible server serves; jev-1.13 is the only valid value. Every other
model is tuned through the store, keyed by model id, so the key has nothing
to say about them.
Measuring a profile: the preregistered eval
The shadow hold is not a dead end. Each use with a committed, labeled take ships a preregistered eval spec: the fixture and its content hash, a seed with a frozen dev/counted split, the promotion metric with its ceiling and minimum n, and the exact threshold values the use reads - all frozen and content-hashed before anything is graded, so a spec edited after the fact invalidates what came before instead of quietly re-scoping the claim.
o2b decision-model eval run --use <use> grades that take over your own
configured adapter. The requests are real spend counted toward the daily
cost gate (the deterministic scorer is canonical; an external judge would
be advisory only). The verdict is a three-arm rule, the same shape the
extract-prefilter report keeps: promote only when the counted error meets
the ceiling, does not regress against the deterministic baseline, and the
counted n is met; a short n is not_enough_evidence, never a pass.
Every run caches a decision_model_eval_grade record in the vault's
continuity log - o2b decision-model eval list prints them (or
o2b decision-model report), and malformed records are skipped by name,
never silently. These records are review evidence. The gate input is a
separate machine-side tuning mark under the state root
(${XDG_STATE_HOME:-~/.local/state}/open-second-brain/decision-model/),
written only by eval run, and only on a promote verdict for a named
profile. With a valid mark, that profile-use pair runs enforce where it
would have run shadow.
The mark is honored only while the installed spec still matches the one it
was measured under: the spec hash, the fixture hash and the pinned
threshold values are all re-checked on every read, along with the endpoint
that earned it (the provider, model and base-url hash recorded in the
mark). Edit the spec, move the fixture, change a threshold, or point the
config at another endpoint, and the pair falls back to shadow - the
fail-closed direction. Deleting the mark does the same; nothing breaks.
o2b decision-model check names the measured state (untuned, baseline,
measured under a spec hash, or measured under a spec hash on another
endpoint - provider, model or base-url mismatch) and lists the cached
grades, and the doctor readiness line carries the same summary. The local
thresholds store ("Thresholds" above) is the other tuning path: it needs
no eval run and no mark, only the use's labelled report data, and it keys
on the model id instead of the profile. Tuning a profile by hand-editing
the source table stays possible and still wins over a mark; the mark only
ever carries pairs the table leaves untuned.