Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
Python agent-evaluationagent-pluginsagent-securityagent-skillsagentic-aibenchmarkci-templatesclaude-codecodexevaluateevaluationharnesspluginplugin-evaluationsecurity-scannerskill-evalskill-evaluationskill-evaluatorskill-gap-analysisskills
Commits on main
fix: target dependency coverage and Tier 3 result integrity (#196)
87274f4 Christopher Kevin committed 1 month ago
fix(tier3): include ATIF content parts in judge answer evidence (#203)
6b0d349 YMuskrat committed 1 month ago
fix(tier3): honor per-step tokens when detecting runtime failures (#199)
45622f8 YMuskrat committed 1 month ago
feat: default to GPT-6.1 Sol and Claude Opus 5.5 across Tiers 1-3 (#188)
03ef951 Narendran Raghavan committed 1 month ago
fix: avoid redundant Node/npm during Docker agent setup (#193)
b57de44 Narendran Raghavan committed 1 month ago
fix: avoid redundant Node/npm during Docker agent setup (#193)
4ac5189 Christopher Kevin committed 1 month ago
feat(report): show the SkillSpector version in the Tier 1 HTML card (#187)
c6ad8e4 Narendran Raghavan committed 1 month ago
docs: bring the published docs in line with v0.5.0 (#186)
ee8ee4c Narendran Raghavan committed 1 month ago
fix(tier3): propagate Harbor dual-arm task suffixes and normalize canonical case ID resolution (#161)
a84ac99 Karl Weinmeister committed 1 month ago
feat(tier3): rebuild ATIF from OpenCode and Codex agent logs (#119)
7d49311 Mohammed Imran Khan committed 1 month ago
feat(tier3): enforce structured JSON schemas and rate-limit retries in LLM judges (#145)
c65e63c Karl Weinmeister committed 1 month ago
feat(tier3): make evidence bundle budgets and final response limit configurable (#160)
90ecf77 Karl Weinmeister committed 1 month ago
fix(tier3): make log JWT redaction linear-time (#176)
e639ece Narendran Raghavan committed 1 month ago
fix(reporting): strip terminal controls from catalog and view output (#175)
6670c90 Narendran Raghavan committed 1 month ago
fix(redaction): make sensitive-key, JWT, and private-key matching linear-time (#172)
a2b01c7 Narendran Raghavan committed 1 month ago
fix(tier3): detect apply_patch writes to protected paths and mask GitHub tokens (#174)
d191bd1 Narendran Raghavan committed 1 month ago
fix(reporting): escape untrusted text in Rich CLI output (#173)
b234e48 Narendran Raghavan committed 1 month ago
fix(security): stop reporting User-Agent versions as IP addresses (#168)
d015741 Jeremy Levartovsky committed 1 month ago
fix(tier3): require invocation evidence for script execution credit (#154)
33eb9dc Maverick committed 1 month ago
fix(tier2): support larger bounded collection scans (#166)
bf87c37 Narendran Raghavan committed 1 month ago
fix: keep --no-llm negative cases off-skill (#100)
1503bde Mohammed Imran Khan committed 1 month ago
Keep fenced code within its Markdown section during chunking (#165)
00c97f8 Sylvester Kaczmarek committed 1 month ago