Skip to content

Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.

docs(tier3): point the NeMo Gym example at an OpenAI Responses endpoint (#615)

Christopher Kevinfc02eb3a4918
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 5e558fe9..8e4c2696 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -298,6 +298,8 @@ All notable changes to SkillEvaluator are documented in this file.
preserves cancellation over cleanup errors, and reaps adopted run-owned servers.
- Native NeMo Gym project skill projections run after authored image instructions
and final workspace resets, even when an earlier layer already copied skills.
+- The NeMo Gym example in the Tier 3 docs points the policy model at an OpenAI
+ Responses endpoint instead of NVIDIA Build, which cannot serve Codex.
- Native NeMo Gym OpenCode evaluations project staged skills into the task workdir
so skill discovery works with Gym's isolated home and configuration directories.
Baseline tasks use the same layout without the target skill.
diff --git a/docs/tier3-live-evaluation.mdx b/docs/tier3-live-evaluation.mdx
index aa8ceecb..294ac2d2 100644
--- a/docs/tier3-live-evaluation.mdx
+++ b/docs/tier3-live-evaluation.mdx
@@ -727,8 +727,9 @@ a placeholder; policy credentials remain in the model-server environment.
```bash
# Requires a separately verified, pinned Gym task-folder implementation.
export SKILL_EVAL_NEMO_GYM_BIN=/opt/nemo-gym/.venv/bin/gym
-export SKILL_EVAL_NEMO_GYM_MODEL_URL=https://integrate.api.nvidia.com/v1
-export SKILL_EVAL_NEMO_GYM_MODEL_API_KEY="$NVIDIA_API_KEY"
+# Codex needs an endpoint that serves the OpenAI Responses API; NVIDIA Build does not.
+export SKILL_EVAL_NEMO_GYM_MODEL_URL=https://api.openai.com/v1
+export SKILL_EVAL_NEMO_GYM_MODEL_API_KEY="$OPENAI_API_KEY"
skillevaluator tier3 doctor --backend nemo-gym --agents codex
skillevaluator tier3 evaluate ./my-skill --backend nemo-gym --agents codex --model <model>
```