The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu
Go agent-ai-cliagent-harnessesagent-securitybrowser-extensionciclaude-codecloudcodexcursordetection-engineeringjevknowledge-basememorymobile-device-managementobservabilitysecuritysecurity-information-and-event-managementsecurity-toolstelemetrytraces
Honour BEACON_JEV_ENDPOINT and BEACON_JEV_MODEL in memory evaluations run (#651)
Justin D'Souza188c44b80717
diff --cc docs/cli/memory.mdxindex 892ddbc6,13625933..6ea1084c--- a/docs/cli/memory.mdx+++ b/docs/cli/memory.mdx@@@ -59,21 -59,24 +59,35 @@@ Useful flags- `--timeout <duration>` sets the Jev request timeout.- `--json` prints machine-readable output.- `--jev-api-key` defaults to `TYPESAFE_API_KEY`, then `BEACON_JEV_API_KEY`.- `--jev-endpoint` can also come from `BEACON_JEV_ENDPOINT`, and the model can come- from `BEACON_JEV_MODEL`.+ Each Jev setting is resolved from the flag first, then the environment, then the+ built-in default:++ - `--jev-api-key` defaults to `TYPESAFE_API_KEY`, then `BEACON_JEV_API_KEY`.+ - `--jev-endpoint` defaults to `BEACON_JEV_ENDPOINT`, then the hosted TypeSafe+ endpoint `https://api.typesafe.ai/v1/systemone`.+ - `--jev-model` defaults to `BEACON_JEV_MODEL`, then `jev-latest`.++ This is how you point Beacon at a TypeSafe-compatible gateway. For example, to+ evaluate through Vercel AI Gateway with an AI Gateway API key:++ ```bash+ export TYPESAFE_API_KEY=<AI Gateway API key>+ export BEACON_JEV_ENDPOINT=https://ai-gateway.vercel.sh/typesafe/v1/systemone+ export BEACON_JEV_MODEL=typesafe-ai/jev+ beacon memory evaluations run --limit 10+ ```+Each completed evaluation stores a `score`, the plain mean of the three rubric+probabilities. A trace becomes a review candidate only when both hold:++- `task_success` is at least `0.50`. This is a precondition, so a trace the+ evaluator judged as failed is never promoted, however high the other two+ answers are.+- `score` is at least `0.60`.++Text output lists each completed evaluation that was not promoted with the+reason, for example `Not promoted: eval_... (task_success 0.27 is below 0.50)`.+List and inspect stored evaluations:```bash