Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
Python agent-evaluationagent-pluginsagent-securityagent-skillsagentic-aibenchmarkci-templatesclaude-codecodexevaluateevaluationharnesspluginplugin-evaluationsecurity-scannerskill-evalskill-evaluationskill-evaluatorskill-gap-analysisskills
Commits on main
fix(security): make the SkillSpector timeout configurable (#271)
54fff6b Narendran Raghavan committed 18 days ago
fix(tier2): report a chat-model refusal as an advisory dedup finding (#281)
e093866 Narendran Raghavan committed 18 days ago
fix: Git output decoding, PowerShell hook conditions and shared verifier redaction (#260 follow-ups) (#312)
b6e5d49 Narendran Raghavan committed 18 days ago
feat(tier1): keep SkillSpector LLM provenance in reports (#298)
5adf3d1 Narendran Raghavan committed 18 days ago
fix(security): keep SkillSpector's nv_build model registry for Build scans (#288)
d82cfe0 Narendran Raghavan committed 18 days ago
feat(tier3): add opt-in --max-cases limit for exploratory runs (#283)
6d5d57f Narendran Raghavan committed 19 days ago
fix(tier2): report context-deduplication work limits as incomplete (#280)
d596ec3 Narendran Raghavan committed 19 days ago
fix(tier3): launch the Bubblewrap binary that local mode detected (#278)
1836cd5 Narendran Raghavan committed 19 days ago
fix(tier3): write generated test.sh and grader starters with LF endings (#266)
3306937 Narendran Raghavan committed 19 days ago
fix(tier3): disable Claude Code background tasks in eval trials (#268)
1e147a8 Narendran Raghavan committed 19 days ago
fix(reporting): stop labeling recorded Tier 3 baselines as missing (#285)
a546a9c Narendran Raghavan committed 19 days ago
fix(quality): report the 5,000-token skill budget as advisory (#272)
ffbbd6b Narendran Raghavan committed 19 days ago
fix(security): redact credentials in declared component paths (#301)
c8e26ce Narendran Raghavan committed 21 days ago
fix: address review findings from #249 and #250 (#260)
63d2b31 Narendran Raghavan committed 21 days ago
ci: run main test jobs in parallel (-n auto) (#261)
d581884 Narendran Raghavan committed 22 days ago
fix(tier3): retain scores after verified timeout recovery (#222)
fad61ed Christopher Kevin committed 23 days ago
Share trace evidence, applicable scores and security checks (#258)
239e321 Christopher Kevin committed 22 days ago
Share trace evidence, applicable scores and security checks (#258)
74cb19b Narendran Raghavan committed 22 days ago
Add reusable GitLab skill evaluation and repository onboarding (#248)
b63288c Narendran Raghavan committed 22 days ago
Add reusable GitLab skill evaluation and repository onboarding (#248)
34d84cd Christopher Kevin committed 22 days ago
Compose private inventory, audits and catalog checks (#257)
aca88a9 Narendran Raghavan committed 22 days ago
Add opt-in Unicode character facts for Tier 3 judges (#242)
58953e9 Narendran Raghavan committed 22 days ago
Add opt-in Unicode character facts for Tier 3 judges (#242)
abdf333 mavericksea-ai committed 25 days ago
fix(tier2): bound embedding chunks and preserve source evidence (#227)
d229da7 Narendran Raghavan committed 22 days ago
fix(tier2): bound embedding chunks and preserve source evidence (#227)
ee3ecab Christopher Kevin committed 23 days ago
fix(reporting): retain completed-agent results in Tier 3 reports (#223)
b3a1706 Narendran Raghavan committed 22 days ago