Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
Python agent-evaluationagent-pluginsagent-securityagent-skillsagentic-aibenchmarkci-templatesclaude-codecodexevaluateevaluationharnesspluginplugin-evaluationsecurity-scannerskill-evalskill-evaluationskill-evaluatorskill-gap-analysisskills
---
title: "SkillEvaluator"
description: "Check an agent skill, measure its effect on a real agent, or add automatic checks to your repository."
layout: overview
---
Check your agent skills for quality and security, then measure whether they help an agent complete real tasks.
<div id="try-it-in-two-minutes">
## Start with a first result
<Card title="Run your first check" icon="rocket" href="/skills/skillevaluator/quickstart">
**Install → check the example → read your result.**
Use a bundled skill to get a quality score and actionable findings.
The quickstart links to installation and walks through a bundled skill.
No API key, GPU, or Docker required for this check.
</Card>
</div>
## What do you want to do?
<CardGroup cols={3}>
<Card title="Test a skill with an agent" icon="flask" href="/skills/skillevaluator/tier3-live-evaluation">
Run the same tasks with and without your skill. Open a report comparing the results.
</Card>
<Card title="Add checks to your repository" icon="code-branch" href="/skills/skillevaluator/repository-integration">
Copy a GitHub or GitLab template. Get automatic quality and security reports when skills change.
</Card>
<Card title="Evaluate from your agent" icon="robot" href="/skills/skillevaluator/agent-skill">
Install the SkillEvaluator skill and ask your agent to run an evaluation.
</Card>
</CardGroup>
## The three tiers
Choose the check that answers your question. Each tier can run independently.
| Your question | Evaluation |
| --- | --- |
| Is my skill well-formed and safe to use? | [Tier 1: Validation](/skills/skillevaluator/tier1-validation) |
| Will my agent load the skill correctly? | [Tier 1: Harness Check](/skills/skillevaluator/harness-check) |
| Does it repeat or overlap with existing guidance? | [Tier 2: Deduplication](/skills/skillevaluator/tier2-deduplication) |
| Does it improve an agent's results? | [Tier 3: Live Agent Evaluation](/skills/skillevaluator/tier3-live-evaluation) |
Need a specific option? Open [Installation](/skills/skillevaluator/installation),
[Providers & Credentials](/skills/skillevaluator/configuration),
[Reports & Results](/skills/skillevaluator/reports), or the
[CLI Reference](/skills/skillevaluator/cli-reference).
<Accordion title="About SkillEvaluator and support">
SkillEvaluator is an open-source, multi-tier framework for evaluating AI agent artifacts,
starting with agent skills: deterministic quality gates, semantic overlap detection,
synthetic eval dataset generation, and live agent evaluation.
It is part of [NVIDIA Verified Skills](https://docs.nvidia.com/skills/) and the
[NVIDIA/skills catalog](https://github.com/NVIDIA/skills). Tier 1 integrates
[SkillSpector](https://github.com/NVIDIA/SkillSpector) for security scanning.
**Experimental:** community-supported on a best-effort basis, with no SLA.
[Get support](https://github.com/NVIDIA/SkillEvaluator/blob/main/SUPPORT.md),
[report a security vulnerability privately](https://github.com/NVIDIA/SkillEvaluator/blob/main/SECURITY.md),
or [contribute](/skills/skillevaluator/developer-guide).
SkillEvaluator is [Apache-2.0 licensed](https://github.com/NVIDIA/SkillEvaluator/blob/main/LICENSE).
</Accordion>