Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
SkillEvaluator CI examples
These examples live in the public NVIDIA/SkillEvaluator repository. You do not need access to a personal demo repository, private pipeline or hosted report to view them. For setup, copy a template.
Report preview

This captured example shows a skill scoring 98.2/100 against a 100-point threshold. The scan fails because it is below the configured threshold. Quality score and scan verdict are separate; security and other required checks also affect the verdict. This preview is an example, not a result for your repository.
GitHub and GitLab retain the HTML report as a job artifact. Open index.html in
the downloaded artifact, then select a skill for its detailed findings.
GitHub Actions summary
A shortened illustrative example:
Tier 1 scan result: FAILED
1 skill · 0 passed · 1 failed
Problem: Quality score 98.2/100 is below the required 100/100
The full report identifies the quality checks that need attention. Improve the skill to meet the repository's chosen threshold, then submit the change again.
Optional GitHub PR comment
The following is a shortened illustrative comment, not a live pull request:
github-actions[bot]
Tier 1 scan result: FAILED
1 skill · 0 passed · 1 failed · 0 incomplete
Skill Status Quality / 100 skills/glossary-builderFAIL 98.2 / A Problem: Quality score 98.2/100 is below the required 100/100.
Suggested fix: Open the skill report and address the quality findings.
Open full report · View workflow run
The last line shows link labels only. In an actual comment, they link to your repository's hosted report and workflow run. The publisher updates one comment on an eligible current same-repository PR after successful report publication. Follow the optional publisher setup.
GitLab job log
A shortened excerpt corresponding to the report preview:
Skill result: "skills/glossary-builder" — FAILED
Quality: 98.2/100, grade "A"
Check: "QUALITY" — "failed"
Scanner error: "Quality score 98.2/100 is below the required 100/100 ..."
Download skill-evaluator-report from the job for the complete findings. The
GitLab integration provides job artifacts; automatic MR comments and hosted
reports are not included.