Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
Python agent-evaluationagent-pluginsagent-securityagent-skillsagentic-aibenchmarkci-templatesclaude-codecodexevaluateevaluationharnesspluginplugin-evaluationsecurity-scannerskill-evalskill-evaluationskill-evaluatorskill-gap-analysisskills
Merge branch 'main' into naren/rollout-benchmark-pr25
Narendran Raghavand39eb52cd85f
diff --cc src/skillevaluator/constants.pyindex 2a207650,f4e10a1e..5926ba41--- a/src/skillevaluator/constants.py+++ b/src/skillevaluator/constants.py@@@ -565,10 -565,10 +567,10 @@@ TOKEN_EFFICIENCY_EVALUATOR_NAME: str =# LLM-AS-JUDGE: TIER 3 DIMENSION + INSIGHTS# =============================================================================- DIMENSION_JUDGE_MODEL = "gpt-5.4-mini"- DIMENSION_JUDGE_MAX_TOKENS = 2048+ DIMENSION_JUDGE_MODEL = CHAT_DEFAULT_OPENAI+ DIMENSION_JUDGE_MAX_TOKENS = 4096DIMENSION_JUDGE_TEMPERATURE: float | None = 0.0-DIMENSION_VERDICT_PASS_THRESHOLD = 0.7+DIMENSION_VERDICT_PASS_THRESHOLD = 0.5DIMENSION_VERDICT_NEUTRAL_THRESHOLD = 0.4# LLM-as-Judge for the Insights tab (additional Conclusions and Recommendations