Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
Python agent-evaluationagent-pluginsagent-securityagent-skillsagentic-aibenchmarkci-templatesclaude-codecodexevaluateevaluationharnesspluginplugin-evaluationsecurity-scannerskill-evalskill-evaluationskill-evaluatorskill-gap-analysisskills
Commits on main
feat(ci): onboard shared components and private report sites (#335)
f878f99 Christopher Kevin committed 17 days ago
feat(tier3): add reproducible local-catalog decoy selection (#263)
2eb1200 YMuskrat committed 17 days ago
fix(ci): publish reports through approved reusable workflow wrappers (#330)
acfb793 Christopher Kevin committed 17 days ago
fix(tier3): stop flagging commands that only name a network client (#309)
f57f15a Narendran Raghavan committed 17 days ago
fix(tier2): derive the intra-skill scalar budget from the chunk ceiling (#270)
300983a Narendran Raghavan committed 17 days ago
fix(tier3): omit local Codex credential dirs from stop-on-pass merges (#276)
cb52c19 Narendran Raghavan committed 17 days ago
fix(tier3): keep custom_only grading policy on unscored conditions (#287)
9d131bd Narendran Raghavan committed 17 days ago
feat(tier1): accept a list of values for metadata.domain (#279)
9e34dd4 Narendran Raghavan committed 17 days ago
fix(tier3): show which Tier 3 source checks failed (#290)
d69c438 Narendran Raghavan committed 17 days ago
fix(tier2): stop catalog queries from matching a skill against its own entry (#296)
9f520db Narendran Raghavan committed 17 days ago
fix(tier3): stop grading force-only rm cleanup as destructive (#265)
55ed08e Narendran Raghavan committed 17 days ago
Add private MCP execution proof and behavior signals (#321)
d31f1a4 Narendran Raghavan committed 17 days ago
fix(reporting): keep benchmark uplift consistent with displayed scores (#269)
8532d57 Narendran Raghavan committed 17 days ago
docs(ci): consolidate template downloads into the repository (#328)
847bdb4 Christopher Kevin committed 18 days ago
feat(ci): ship guided GitHub onboarding in SkillEvaluator 0.7.1 (#327)
3da8c57 Christopher Kevin committed 18 days ago
fix(ci): show concise PR feedback with fresh HTML report links (#326)
46a8cf2 Christopher Kevin committed 18 days ago
fix(ci): require Semgrep in scanner subprocesses (#324)
e4cda13 Christopher Kevin committed 18 days ago
Set up Tier 1 scanners on first use and add --skip (#318)
a705bb6 Christopher Kevin committed 18 days ago
chore(release): reconcile published 0.7.0 metadata (#314)
f23a6f6 Christopher Kevin committed 18 days ago
Show per-case Tier 3 results and a "what to fix" list (#316)
2951548 Christopher Kevin committed 18 days ago
Offer to install a missing agent CLI in local mode (#317)
58e617c Christopher Kevin committed 18 days ago
Private runtime: prepared packages, native loading and evidence over the existing engine (#264)
98556a6 Christopher Kevin committed 18 days ago
fix(docs): render README images in PyPI descriptions (#323)
7696c71 Christopher Kevin committed 18 days ago
fix(security): handle read-only passwd mounts with SkillSpector 2.13.3 (#289)
ffa78ba Narendran Raghavan committed 18 days ago
fix(tier3): write plain progress heartbeats once a minute (#274)
aba2fa1 Narendran Raghavan committed 18 days ago
test(security): cover SkillSpector watchdog duration boundaries (#319)
6e3c7f4 Narendran Raghavan committed 18 days ago
fix(security): accept verified referenced images in partial SkillSpector scans (#299)
f4287e3 Narendran Raghavan committed 18 days ago
feat(tier3): add --skip-with-skill for baseline-only runs (#284)
bfd477b Narendran Raghavan committed 18 days ago
fix: decode Git roots as file names and keep PowerShell prefix findings (#312 review) (#320)
8939d52 Narendran Raghavan committed 18 days ago
feat(doctor): add Tier 1 scanner and Tier 2 provider readiness checks (#293)
77fdf74 Narendran Raghavan committed 18 days ago