Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
Python agent-evaluationagent-pluginsagent-securityagent-skillsagentic-aibenchmarkci-templatesclaude-codecodexevaluateevaluationharnesspluginplugin-evaluationsecurity-scannerskill-evalskill-evaluationskill-evaluatorskill-gap-analysisskills
Commits on main
feat(tier1): score tier 1 quality checks via a configurable YAML file (#213)
830fabe Weston Platter committed 2 months ago
docs: simplify first runs with keyless checks and NVIDIA Build (#218)
3fc1bc1 Christopher Kevin committed 1 month ago
Pass Windows program dirs and Python UTF-8 mode to Harbor (#220)
9ffee8f Godfred Arhinful committed 1 month ago
fix(quality): keep minimum score failures after policy changes (#211)
0c1c7b3 YMuskrat committed 1 month ago
feat(tier3): let installed packages add custom execution backends (#215)
7adf7f1 Roshni Malani committed 1 month ago
fix(judge): treat an "unavailable" schema option as a schema rejection
72f381f Max Freedom Pollard committed 1 month ago
ci: cover every workflow with the hardening guards, and fix the gaps they surface (#126)
aaeb3d4 berzan93-prog committed 1 month ago
fix(tier3): mark never-activated skill evaluations inconclusive (#209)
589dca3 Narendran Raghavan committed 1 month ago
fix(tier3): mark never-activated skill evaluations inconclusive (#209)
efb5ab4 Christopher Kevin committed 1 month ago
feat(reporting): add quantitative skill performance dashboard (#179)
526dd30 Narendran Raghavan committed 1 month ago
feat(reporting): add quantitative skill performance dashboard (#179)
59bd222 Christopher Kevin committed 1 month ago
feat(tier3): support Claude Code on Agent Platform and add active preflight probe (#140)
39df294 Narendran Raghavan committed 1 month ago
feat(tier3): support Claude Code on Agent Platform and add active preflight probe (#140)
35b8da1 Karl Weinmeister committed 1 month ago
feat(tier3): expose Harbor's Runta and Mosaic backends (#183)
7c8136f Narendran Raghavan committed 1 month ago
fix(reporting): require complete tier evidence for publication (#77)
75ce3f0 Narendran Raghavan committed 1 month ago
fix(reporting): require complete tier evidence for publication (#77)
4bc7919 Christopher Kevin committed 2 months ago
fix(tier3): exclude failed skill reads from execution credit (#201)
2e152d9 Narendran Raghavan committed 1 month ago
fix(tier3): exclude failed skill reads from execution credit (#201)
c151df4 YMuskrat committed 1 month ago
fix(tier3): derive accuracy scores from criterion verdicts (#190)
db819c1 Narendran Raghavan committed 1 month ago
fix(tier3): derive accuracy scores from criterion verdicts (#190)
ce3c523 mavericksea-ai committed 1 month ago
feat(tier3): run hardened Docker trials under Kata Containers (#182)
004a49d Narendran Raghavan committed 1 month ago
fix: target dependency coverage and Tier 3 result integrity (#196)
67629d3 Narendran Raghavan committed 1 month ago