Skip to content

The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu

Honour BEACON_JEV_ENDPOINT and BEACON_JEV_MODEL in memory evaluations run (#651)

Justin D'Souza188c44b80717
diff --cc docs/cli/memory.mdx
index 892ddbc6,13625933..6ea1084c
--- a/docs/cli/memory.mdx
+++ b/docs/cli/memory.mdx
@@@ -59,21 -59,24 +59,35 @@@ Useful flags
- `--timeout <duration>` sets the Jev request timeout.
- `--json` prints machine-readable output.
- `--jev-api-key` defaults to `TYPESAFE_API_KEY`, then `BEACON_JEV_API_KEY`.
- `--jev-endpoint` can also come from `BEACON_JEV_ENDPOINT`, and the model can come
- from `BEACON_JEV_MODEL`.
+ Each Jev setting is resolved from the flag first, then the environment, then the
+ built-in default:
+
+ - `--jev-api-key` defaults to `TYPESAFE_API_KEY`, then `BEACON_JEV_API_KEY`.
+ - `--jev-endpoint` defaults to `BEACON_JEV_ENDPOINT`, then the hosted TypeSafe
+ endpoint `https://api.typesafe.ai/v1/systemone`.
+ - `--jev-model` defaults to `BEACON_JEV_MODEL`, then `jev-latest`.
+
+ This is how you point Beacon at a TypeSafe-compatible gateway. For example, to
+ evaluate through Vercel AI Gateway with an AI Gateway API key:
+
+ ```bash
+ export TYPESAFE_API_KEY=<AI Gateway API key>
+ export BEACON_JEV_ENDPOINT=https://ai-gateway.vercel.sh/typesafe/v1/systemone
+ export BEACON_JEV_MODEL=typesafe-ai/jev
+ beacon memory evaluations run --limit 10
+ ```
+Each completed evaluation stores a `score`, the plain mean of the three rubric
+probabilities. A trace becomes a review candidate only when both hold:
+
+- `task_success` is at least `0.50`. This is a precondition, so a trace the
+ evaluator judged as failed is never promoted, however high the other two
+ answers are.
+- `score` is at least `0.60`.
+
+Text output lists each completed evaluation that was not promoted with the
+reason, for example `Not promoted: eval_... (task_success 0.27 is below 0.50)`.
+
List and inspect stored evaluations:
```bash