Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
Python agent-evaluationagent-pluginsagent-securityagent-skillsagentic-aibenchmarkci-templatesclaude-codecodexevaluateevaluationharnesspluginplugin-evaluationsecurity-scannerskill-evalskill-evaluationskill-evaluatorskill-gap-analysisskills
Commits on main
fix(security): show where SkillSpector parse limits apply in incomplete scans (#789)
798031f Matt Van Horn committed 13 hours ago
fix(paths): preserve whitespace in Git repository roots (#743)
85f39de Afloat committed 14 hours ago
fix(tier3): exit nonzero when tier3 validate reports a missing evals/ (#797)
2a117a8 Christopher Kevin committed 14 hours ago
perf(embedding): skip provider calls when queries have no candidates (#720)
7637e1f Afloat committed 15 hours ago
perf(tier3): reuse bootstrap means for constant case deltas (#721)
75380ed Afloat committed 15 hours ago
docs(discovery): clarify aliases into excluded directories (#785)
771a390 YMuskrat committed 15 hours ago
docs: add a focused NeMo Gym integration guide (#772)
ac9fc0e Christopher Kevin committed 17 hours ago
perf(results): reconcile independent diagnostics with local indexes (#761)
a3bb7e9 Afloat committed 17 hours ago
fix(doctor): suggest scanner installs that work on the host platform (#796)
3ecd69b sm18lr88 committed 18 hours ago
fix(tier3): flag HTTP uploads from interpreter code, xh, curl -K and .wgetrc (#771)
620ac2d Narendran Raghavan committed 1 day ago
fix(tier3): flag recursive rm run through interpreter argv lists (#766)
b43c4c5 Narendran Raghavan committed 1 day ago
docs: clarify keyless validation and repository CI setup (#782)
affe1ac Christopher Kevin committed 1 day ago
fix(tier3): keep malformed optional usage metadata unavailable (#725)
e3ede73 Afloat committed 1 day ago
fix(tier3): read rm run through eval, env -S, git grep pagers and quoted names (#765)
9488c95 Narendran Raghavan committed 1 day ago
docs: explain CI setup and link templates in README (#778)
33d49eb Christopher Kevin committed 1 day ago
fix(ci): stop recording repository evaluations as deployments (#779)
2ca66b7 Christopher Kevin committed 1 day ago
fix(ci): allow evaluation jobs to opt out of deployment records (#777)
bf44a24 Christopher Kevin committed 1 day ago
fix(packaging): link PyPI documentation to NVIDIA docs (#775)
b308556 Christopher Kevin committed 1 day ago
chore(release): pin the verified v1.0.1 CI bundle (#774)
914c15c Christopher Kevin committed 2 days ago
fix(release): qualify runtime fixes and prepare v1.0.1
c9369a7 Christopher Kevin committed 2 days ago
fix(api-caller): share one URL scheme check between scripts (#767)
ab7f355 Christopher Kevin committed 2 days ago
fix(create-custom-grader): ship the fixtures its eval cases ask for (#768)
f404bbc Christopher Kevin committed 2 days ago
fix(discovery): scope exclusions below the requested scan root (#746)
13f224a Afloat committed 2 days ago
fix(tier3): turn off OpenCode snapshots to stop with-skill EPIPE exits (#763)
046290d SUHAN RAMANI committed 2 days ago
docs(release): document 1.0.0 changes and upgrade notes (#769)
87bf147 Narendran Raghavan committed 2 days ago
feat(tier1): start yaml setup for Efficiency, Discoverability and Correctness (#611)
0f3d980 Weston Platter committed 2 days ago
fix(lint): avoid recursion failures in valid Python expressions (#760)
6e85d9b Afloat committed 2 days ago