Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
Python agent-evaluationagent-pluginsagent-securityagent-skillsagentic-aibenchmarkci-templatesclaude-codecodexevaluateevaluationharnesspluginplugin-evaluationsecurity-scannerskill-evalskill-evaluationskill-evaluatorskill-gap-analysisskills
Commits on main
fix(reporting): retain completed-agent results in Tier 3 reports (#223)
df16c85 Christopher Kevin committed 23 days ago
Add private manifest discovery and configuration checks (#250)
faa7499 Christopher Kevin committed 22 days ago
fix(tier3): avoid destructive scoring for plain cache cleanup (#228)
fab5a1b Christopher Kevin committed 22 days ago
perf(ci): faster Tier 1 scans and a clearer HTML report (#256)
1aee929 Narendran Raghavan committed 22 days ago
feat(tier3): case-first CLI summary and case-based report findings (#255)
0b12f05 Narendran Raghavan committed 22 days ago
feat(tier3): case-first CLI summary and case-based report findings (#255)
ff1ff7d Christopher Kevin committed 22 days ago
fix(tier3): preserve local verifier uploads and shared step files (#239)
c5e732b Christopher Kevin committed 23 days ago
Share deterministic grading and add bounded input readers (#249)
9df976a Christopher Kevin committed 23 days ago
fix(rubric): preserve source and complete linked guidance (#254)
26d2b0d Narendran Raghavan committed 22 days ago
feat(ci): add reusable Tier 1 scans and GitHub onboarding (#238)
ad4b895 Narendran Raghavan committed 23 days ago
feat(ci): add reusable Tier 1 scans and GitHub onboarding (#238)
376180b Christopher Kevin committed 23 days ago
fix(tier3): preserve local Claude Code instruction variables (#252)
0f5bc0d Narendran Raghavan committed 23 days ago
fix(reporting): preserve unavailable Tier 3 evaluator scores (#251)
5d2a505 Narendran Raghavan committed 23 days ago
docs(cli): compare direct tier workflows with validate --tiers (#237)
da02abd Christopher Kevin committed 23 days ago
fix(tier3): count explicit root searches as wasted tool calls (#229)
eaa68be Christopher Kevin committed 23 days ago
test: wait for descendant exit after streamed exec cleanup (#247)
123b28c Narendran Raghavan committed 23 days ago
fix(pii): recognize reserved placeholder email domains (#246)
dd35391 Narendran Raghavan committed 23 days ago
fix(tier3): gate nested cleanup findings on symlink evidence (#232)
d7514a3 YMuskrat committed 25 days ago
fix(tier3): gate nested cleanup findings on symlink evidence (#232)
64aac04 YMuskrat committed 26 days ago
fix(tier3): flag deletions that escape temporary paths (#230)
0bd92cf YMuskrat committed 24 days ago
fix(tier3): flag deletions that escape temporary paths (#230)
58a5047 YMuskrat committed 27 days ago
fix(quality): score non-mapping metadata instead of crashing (#226)
1c435f7 Max Freedom Pollard committed 25 days ago
Fix Windows subprocess decoding, reasoning-model token budget, and harbor bin resolution (#194)
8cd6555 Narendran Raghavan committed 1 month ago
Fix Windows subprocess decoding, reasoning-model token budget, and harbor bin resolution (#194)
3cb0757 Minervaowl7 committed 1 month ago
fix(tier3): score unrecovered failures and align recovery verdicts (#217)
cf23b0d Narendran Raghavan committed 1 month ago
fix(tier3): score unrecovered failures and align recovery verdicts (#217)
697bc44 Maverick committed 1 month ago
fix(tier3): score unrecovered failures and align recovery verdicts (#217)
82094b4 priyadarshnisundararajan committed 2 months ago
docs: refresh evaluation guidance and expose dashboard guide (#224)
2f27723 Narendran Raghavan committed 1 month ago
feat(tier1): score tier 1 quality checks via a configurable YAML file (#213)
eb77715 Weston Platter committed 1 month ago