The Open Framework for Building Agents
issue: "TBD (maintainer-requested research; no matching issue found)" status: implemented last_updated: "2026-10-06"
Decision model routing in eve
Recommendation
Expose automatic model selection as auto from eve/models and standalone
decide from eve/ai. Build both on AI SDK's experimental_decide API.
decide accepts the SDK's state, typed questions, and request options, with an
optional model defaulting to typesafe-ai/jev. It returns the SDK's typed answers
and response metadata.
auto calls the shared decide wrapper. Model strings use the configured AI
SDK default provider; without an override, the wrapper resolves eve's local
Gateway connection when available and otherwise leaves Gateway resolution to the
SDK. Explicit provider instances retain their own authentication.
Tool authors can call decide with structured state and pass ctx.abortSignal.
Application code can use it without an active eve session. Standalone calls are
not cached; durable per-turn selection remains specific to auto.
AI SDK's decision model specification remains experimental and can change in patch releases. The implemented authoring API is documented in Automatic Model Selection.
import { defineAgent } from "eve";
import { auto } from "eve/models";
export default defineAgent({
model: auto({
options: {
"openai/gpt-5.6-sol": "Difficult reasoning and engineering tasks",
"openai/gpt-5.6-luna": "Routine tasks where fast completion matters",
},
}),
});
Why AI SDK is the boundary
AI SDK 7.0.128 provides experimental_decide (shipped as experimental_evaluate
in 7.0.105 and since renamed) and the v4 experimental decision model contract. The core function accepts a string or
Experimental_DecisionModel, validates input and output, checks supported
question types, retries transient provider failures, propagates cancellation,
and normalizes usage and response metadata.
AI SDK decision.
This gives decision models the same selection rules as other AI SDK models:
- A string resolves through
globalThis.AI_SDK_DEFAULT_PROVIDER. With no override, AI SDK uses Vercel AI Gateway. Gateway credentials and OIDC remain AI SDK concerns. - A provider model instance calls that provider directly. Users install the provider package and configure its credentials.
- A registry or custom provider can supply application aliases without eve learning provider names.
- Provider validation, retries, errors, and model resolution retain AI SDK
semantics. eve forwards the active turn's
AbortSignal.
The boundary removes duplicate request validation, error translation, retries, credential precedence, Gateway URL construction, TypeSafe wire conversion, and provider metadata parsing from eve.
flowchart LR
P[Incoming prompt] --> A[auto]
A --> E[AI SDK decide]
E -->|string ID| G[Default provider / Gateway]
E -->|model instance| D[Installed provider]
G --> C[Choice answer]
D --> C
C --> K[Allowlisted option key]
K --> L[Language model string or instance]
What Jev provides
TypeSafe describes Jev as a System One decision model trained with Reinforcement Learning for Calibrated Decisions. It decides structured state against constrained questions instead of generating prose. The native TypeSafe API describes three primitives:
| TypeSafe primitive | AI SDK type | Result | Useful eve applications |
|---|---|---|---|
| Choice | choice |
One option and, when available, a distribution | Model selection, classification, candidate selection |
| Score | score |
A position on an ordered rubric and optional distribution | Relevance, severity, quality |
| Noul | boolean |
Estimated probability of true | Independent flags and evidence checks |
Sources: System One, primitives, Choice, Score, and Noul.
Every question in one request sees the same state and is decided independently by native Jev. Question IDs correlate inputs and outputs but are not model instructions. A question that targets one item in structured state must identify that item in its instructions. Related questions can share one request; unrelated tenant data should remain in separate requests. State and question authoring.
Choice and Score distributions are optional in the AI SDK contract because language-model adapters do not produce them. Boolean probability is required. TypeSafe's separate confidence statistic remains provider metadata. A generic eve router must therefore make no confidence-threshold promise across providers.
Constrained output prevents fabricated option names after SDK validation, but it does not guarantee the selected option is correct. Model descriptions and labeled decision data remain part of the routing policy. Calibration describes behavior across a dataset rather than certainty for one answer. Confidence.
Provider behavior
AI SDK documents native TypeSafe decision through
typeSafeAi.decisionModel("jev-latest"). OpenAI, Anthropic, and Google expose
decision model adapters that use structured language-model output. Those
adapters decide all questions in one prompt, do not reproduce Jev's native
independent-question execution, and omit Choice and Score distributions.
The Gateway model catalog uses
typesafe-ai/jev as the Gateway model ID.
Using it as an AI SDK string keeps Gateway API keys, Vercel OIDC, request headers,
endpoint selection, and future protocol changes inside AI SDK. A direct TypeSafe
setup installs @ai-sdk/typesafe-ai and supplies its model instance. eve does not
pick between those paths or fall back from one provider to another.
TypeSafe advertises low latency and $0.042 per million input tokens with free output tokens. These are vendor claims rather than an eve benchmark or SLA. The parallel-question cookbook reports a large advantage from sending related questions together, but its comparison uses sequential individual calls; concurrent calls narrow the time difference. Launch and methodology and parallel questions.
eve lifecycle
Prompt-aware routing belongs at step.started. A turn.started resolver now
receives the incoming message and history, but its result must be durable and
cannot contain a live provider model instance. The tool loop resolves the step
model after projecting the turn input and before language-model inference, so it
has the same prompt and can return either a model string or a live instance.
auto stores only the selected option key and turn ID in a durable
ContextKey. A selection is reused for later tool-loop steps in that turn.
Provider model objects stay in authored configuration and are resolved again from
the key after resume. New turns and child sessions make independent choices.
The active execution AbortSignal is passed through dynamic model context into
AI SDK decision. This prevents a cancelled turn from completing routing or
persisting a late choice.
Per-option reasoning is part of the selected dynamic model result, so the
chosen model can override the agent-level reasoning setting. String language
models retain standard Gateway resolution; LanguageModel instances retain
direct provider behavior.
Scope boundaries
eve owns the decide wrapper and its default model and authentication behavior.
AI SDK owns decision schemas, validation, retries, errors, and result metadata.
The integration adds no TypeSafe-specific decide function, direct HTTP client,
provider fallback, confidence threshold, or built-in agent-callable decision tool.
No live Jev inference or independent quality, latency, or cost benchmark was run for this research. Deterministic tests use AI SDK decision model mocks and exercise model selection, validation, cancellation, retention, provider model instances, reasoning, and packaging.