Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
Python agent-evaluationagent-pluginsagent-securityagent-skillsagentic-aibenchmarkci-templatesclaude-codecodexevaluateevaluationharnesspluginplugin-evaluationsecurity-scannerskill-evalskill-evaluationskill-evaluatorskill-gap-analysisskills
docs(tier3): point the NeMo Gym example at an OpenAI Responses endpoint (#615)
Christopher Kevinfc02eb3a4918
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 5e558fe9..8e4c2696 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -298,6 +298,8 @@ All notable changes to SkillEvaluator are documented in this file.preserves cancellation over cleanup errors, and reaps adopted run-owned servers.- Native NeMo Gym project skill projections run after authored image instructionsand final workspace resets, even when an earlier layer already copied skills.+- The NeMo Gym example in the Tier 3 docs points the policy model at an OpenAI+ Responses endpoint instead of NVIDIA Build, which cannot serve Codex.- Native NeMo Gym OpenCode evaluations project staged skills into the task workdirso skill discovery works with Gym's isolated home and configuration directories.Baseline tasks use the same layout without the target skill.diff --git a/docs/tier3-live-evaluation.mdx b/docs/tier3-live-evaluation.mdxindex aa8ceecb..294ac2d2 100644--- a/docs/tier3-live-evaluation.mdx+++ b/docs/tier3-live-evaluation.mdx@@ -727,8 +727,9 @@ a placeholder; policy credentials remain in the model-server environment.```bash# Requires a separately verified, pinned Gym task-folder implementation.export SKILL_EVAL_NEMO_GYM_BIN=/opt/nemo-gym/.venv/bin/gym-export SKILL_EVAL_NEMO_GYM_MODEL_URL=https://integrate.api.nvidia.com/v1-export SKILL_EVAL_NEMO_GYM_MODEL_API_KEY="$NVIDIA_API_KEY"+# Codex needs an endpoint that serves the OpenAI Responses API; NVIDIA Build does not.+export SKILL_EVAL_NEMO_GYM_MODEL_URL=https://api.openai.com/v1+export SKILL_EVAL_NEMO_GYM_MODEL_API_KEY="$OPENAI_API_KEY"skillevaluator tier3 doctor --backend nemo-gym --agents codexskillevaluator tier3 evaluate ./my-skill --backend nemo-gym --agents codex --model <model>```