Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
Python agent-evaluationagent-pluginsagent-securityagent-skillsagentic-aibenchmarkci-templatesclaude-codecodexevaluateevaluationharnesspluginplugin-evaluationsecurity-scannerskill-evalskill-evaluationskill-evaluatorskill-gap-analysisskills
Commits on main
fix(calculator): accept signed values and reject trailing input in conversions (#737)
6b0a647 Dhananjay Aggarwal committed 3 days ago
fix(text-analyzer): leave punctuation-only tokens out of the top words (#738)
c43365b Dhananjay Aggarwal committed 3 days ago
fix(task-list): preserve status words inside task descriptions (#680)
ee5cc60 PM committed 3 days ago
docs: simplify installation and guide dataset-first evaluations (#709)
64b237f Christopher Kevin committed 4 days ago
ci: run Windows Tier 2 tests in parallel and align native Windows action pins (#758)
0b08ae1 Christopher Kevin committed 4 days ago
docs(readme): install first, then three ways to use it, and a two-step quickstart (#756)
3ec57ba Christopher Kevin committed 4 days ago
test(tier3): time the masking linearity check on equal work (#717)
d2cb138 Christopher Kevin committed 4 days ago
ci: pin the repository's skill callers to the released evaluator commit (#712)
d99d304 Christopher Kevin committed 4 days ago
feat: add pre-commit hooks for offline Tier 1 skill checks (#690)
8fe9723 Matt Van Horn committed 4 days ago
test(tier3): accept either cooperative cancellation result (#707)
7c4eabd Christopher Kevin committed 4 days ago
fix(cli): validate Tier 1 before preparing scanners (#706)
680e1c0 Christopher Kevin committed 4 days ago
docs(readme): say what get started offers and name the harness check in its heading (#708)
1f481a2 Christopher Kevin committed 4 days ago
ci: take macOS Tier 2 off pull requests and gate the Tier 3 platform jobs by path (#704)
e1b3a21 Christopher Kevin committed 4 days ago
docs(readme): centre the header and add a one-line tagline (#705)
4ecad33 Christopher Kevin committed 4 days ago
docs(readme): highlight the harness check under the get-started table (#703)
ccac29e Christopher Kevin committed 4 days ago
ci: run only the documentation tests for docs, changelog and readme changes (#701)
a9097b6 Christopher Kevin committed 4 days ago
docs(harness): document the JSON block and when the harness check helps (#700)
7c31fad Christopher Kevin committed 4 days ago
docs(harness): show the harness check on its own and inside validate (#699)
c019e6f Christopher Kevin committed 4 days ago
docs(harness): list what the harness check looks at and the harnesses it covers (#698)
02cf4b6 Christopher Kevin committed 4 days ago
docs(harness): show the detailed view and how to fix each finding (#697)
da6d6fc Christopher Kevin committed 4 days ago
docs(harness): add a Harness Check page that shows what the check reports (#696)
d2c5901 Christopher Kevin committed 4 days ago
docs(readme): add GitHub stars and forks badges (#695)
5197f27 Christopher Kevin committed 4 days ago
fix(tier3): show the execution backend in report.html (#694)
eaca723 Christopher Kevin committed 4 days ago
docs(tier3): state that Claude Code has run live on NeMo Gym (#692)
c6e2d02 Christopher Kevin committed 4 days ago
ci: run both HTML parser compatibility versions in one job (#686)
530b119 Christopher Kevin committed 4 days ago
fix(tier3): record the execution backend in the Tier 3 report data (#693)
63c77f6 Christopher Kevin committed 4 days ago
fix(tier3): give the Windows process-tree kill longer than a plain reap (#687)
7f9d2c5 Christopher Kevin committed 4 days ago