Skip to content

armgit / eve

Public Imported from GitHub
0

The Open Framework for Building Agents

docs: link Jev guides and fix Jev capitalization (#3663)

Ben Sabic54fc9a3e95b6
diff --git a/docs/evals/judge.mdx b/docs/evals/judge.mdx
index 7c33e165d..62fd23667 100644
--- a/docs/evals/judge.mdx
+++ b/docs/evals/judge.mdx
@@ -3,7 +3,7 @@ title: "Judge"
description: "Grade evals with evaluation models using criteria, typed questions, or batches, and set thresholds on each assertion."
---
-Use `t.judge(...)` when a deterministic [assertion](./assertions) cannot capture what a good response looks like. It calls [`evaluate` from `eve/ai`](../guides/evaluate#evaluate-inside-a-tool) and defaults to `typesafe-ai/jev`. The judge is separate from the agent under test and is used only for scoring.
+Use `t.judge(...)` when a deterministic [assertion](./assertions) cannot capture what a good response looks like. It calls [`evaluate` from `eve/ai`](../guides/evaluate#evaluate-inside-a-tool) and defaults to `typesafe-ai/jev`, TypeSafe AI's [Jev evaluation model](https://vercel.com/i/what-is-jev). The judge is separate from the agent under test and is used only for scoring.
```ts
import { defineEval } from "eve/evals";
diff --git a/docs/guides/evaluate.md b/docs/guides/evaluate.md
index 622e1eb44..45cb1a3fb 100644
--- a/docs/guides/evaluate.md
+++ b/docs/guides/evaluate.md
@@ -14,7 +14,8 @@ patch releases.
## Choose from Gateway models
-By default, `auto` evaluates with `typesafe-ai/jev`. Like other AI SDK
+By default, `auto` evaluates with `typesafe-ai/jev`, TypeSafe AI's
+[Jev evaluation model](https://vercel.com/i/what-is-jev). Like other AI SDK
model strings, it uses Vercel AI Gateway unless the application has configured a
different global default provider.
@@ -141,7 +142,7 @@ export default defineTool({
The choice above is typed as `"billing" | "support"`. Each question appears under
its authored key in `result.answers`. Results also include token usage, warnings,
-provider metadata, and response metadata. To use that choice to delegate while keeping specialist subagents out of the parent model's tools, see [Route to a hidden subagent with JEV](/docs/tools/workflows#route-to-a-hidden-subagent-with-jev).
+provider metadata, and response metadata. To use that choice to delegate while keeping specialist subagents out of the parent model's tools, see [Route to a hidden subagent with Jev](/docs/tools/workflows#route-to-a-hidden-subagent-with-jev).
`evaluate` accepts AI SDK evaluation options, including `maxRetries`, `headers`,
and `providerOptions`. Pass an `abortSignal` to cancel the request. Input and
@@ -178,7 +179,9 @@ export default defineTool({
The evaluation model reviews the tool name and input for dangerous effects. A
caution, failed review, or incomplete input requires human approval. See
[Human-in-the-loop approvals](/docs/human-in-the-loop#approvals) for classifier
-options and data handling.
+options and data handling, and the
+[Auto-approve tool calls with Jev](https://vercel.com/kb/guide/auto-approve-tool-calls-eve-jev)
+guide for an end-to-end walkthrough.
## Runtime behavior
diff --git a/docs/tools/human-in-the-loop.md b/docs/tools/human-in-the-loop.md
index 3e8b9d6cd..b85147033 100644
--- a/docs/tools/human-in-the-loop.md
+++ b/docs/tools/human-in-the-loop.md
@@ -39,7 +39,7 @@ export default defineTool({
By default, omitted `approval` behaves like `never()`, so tool calls may execute without human approval. Require human approval or other safeguards for sensitive, irreversible, regulated, financial, healthcare, employment, housing, legal, safety-impacting, user-impacting, or external side-effecting actions.
-`auto()` uses an [AI SDK evaluation model](/docs/guides/evaluate) to classify each call as `clear` or `caution`. It defaults to `typesafe-ai/jev`. Like `evaluate`, a model string uses Vercel AI Gateway unless the application configures a global AI SDK default provider:
+`auto()` uses an [AI SDK evaluation model](/docs/guides/evaluate) to classify each call as `clear` or `caution`. It defaults to `typesafe-ai/jev`, TypeSafe AI's [Jev evaluation model](https://vercel.com/i/what-is-jev). Like `evaluate`, a model string uses Vercel AI Gateway unless the application configures a global AI SDK default provider:
```ts
approval: auto({ model: "typesafe-ai/jev" });
diff --git a/docs/tools/workflows.mdx b/docs/tools/workflows.mdx
index 5ec491d3c..bc1d47a11 100644
--- a/docs/tools/workflows.mdx
+++ b/docs/tools/workflows.mdx
@@ -343,9 +343,9 @@ The first argument is an available agent target. On the root agent, `"agent"` st
continue an existing child. An inline `outputSchema` requires structured output and determines the
return type, so `result` in the example is typed as `{ findings: string[] }`.
-### Route to a hidden subagent with JEV
+### Route to a hidden subagent with Jev
-Use an authored workflow tool when the parent model should decide to delegate, but JEV should choose the specialist. Set `tool: false` on each specialist so the parent model sees only the routing tool. The specialists remain available through `ctx.agents` and callable through `ctx.agent()`.
+Use an authored workflow tool when the parent model should decide to delegate, but [Jev](https://vercel.com/i/what-is-jev) should choose the specialist. Set `tool: false` on each specialist so the parent model sees only the routing tool. The specialists remain available through `ctx.agents` and callable through `ctx.agent()`.
If the router may select the root-copy `agent`, author a root `description` that explains which work should stay with a copy of the root. `ctx.agents.agent` always exists in a top-level root workflow, but its `description` is an empty string when omitted from `agent.ts`. `agentRouter()` considers only entries with a non-empty description, so it does not route to an undescribed root copy.
@@ -378,7 +378,7 @@ export default defineAgent({
});
```
-Define the model-visible router under `agent/tools/`. Build the JEV criteria from the subagents' effective descriptions, then pass its typed choice directly to `ctx.agent()`:
+Define the model-visible router under `agent/tools/`. Build the Jev criteria from the subagents' effective descriptions, then pass its typed choice directly to `ctx.agent()`:
```ts title="agent/tools/agent-router.ts"
import { evaluate } from "eve/ai";
@@ -427,7 +427,7 @@ export default defineWorkflowTool({
});
```
-The evaluation runs in a step so workflow replay records its result instead of making the routing request again. The parent model receives `agent-router`, but not `researcher` or `operator`. JEV returns the typed `"researcher" | "operator"` choice, and the selected subagent's result becomes the `agent-router` tool result.
+The evaluation runs in a step so workflow replay records its result instead of making the routing request again. The parent model receives `agent-router`, but not `researcher` or `operator`. Jev returns the typed `"researcher" | "operator"` choice, and the selected subagent's result becomes the `agent-router` tool result.
`ctx.agents` is a replay-stable metadata snapshot taken when the workflow starts. In a top-level root workflow, `ctx.agents.agent` always represents the root-copy target and carries the root's authored `description` when provided. The snapshot also includes declared agents hidden from the parent model with `tool: false` or `disableTool()`, but exposes no model definitions, credentials, or callbacks. A delegated root copy omits `agent`, so `agentRouter()` cannot select another root copy recursively. Invocation still checks each target's availability through `ctx.agent()`.
@@ -441,11 +441,11 @@ import { agentRouter } from "eve/tools/agent-router";
export default agentRouter();
```
-Its input is `{ message: string, outputSchema?: object }`. `agentRouter()` ignores entries without a non-empty description. With two or more described targets, it sends the message and effective descriptions to JEV, then invokes the selected name through `ctx.agent()`. It invokes a sole target without evaluation and forwards an optional `outputSchema` unchanged. In a top-level root session, the candidates include the root-copy `agent`; delegated root copies and declared subagent contexts include only their declared targets.
+Its input is `{ message: string, outputSchema?: object }`. `agentRouter()` ignores entries without a non-empty description. With two or more described targets, it sends the message and effective descriptions to Jev, then invokes the selected name through `ctx.agent()`. It invokes a sole target without evaluation and forwards an optional `outputSchema` unchanged. In a top-level root session, the candidates include the root-copy `agent`; delegated root copies and declared subagent contexts include only their declared targets.
When an authored tool accepts a JSON Schema supplied by the model, represent that input with a permissive object such as `z.looseObject({})`. Avoid `z.record(z.string(), z.json())`: its generated JSON Schema uses `propertyNames`, which OpenAI does not support. Validate the supplied value at the point where your tool consumes it.
-This pattern controls specialist selection, not whether the parent model delegates at all. Route before the parent model runs if every incoming request must go through JEV.
+This pattern controls specialist selection, not whether the parent model delegates at all. Route before the parent model runs if every incoming request must go through Jev.
## Report progress: `yield`