Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
Python agent-evaluationagent-pluginsagent-securityagent-skillsagentic-aibenchmarkci-templatesclaude-codecodexevaluateevaluationharnesspluginplugin-evaluationsecurity-scannerskill-evalskill-evaluationskill-evaluatorskill-gap-analysisskills
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
instances:
- url: nvidia-skillevaluator.docs.buildwithfern.com/skills/skillevaluator
custom-domain: docs.nvidia.com/skills/skillevaluator
multi-source: true
title: SkillEvaluator Documentation
global-theme: nvidia
navbar-links:
- type: github
value: https://github.com/NVIDIA/SkillEvaluator
logo:
light: ../docs/assets/skillevaluator-logo.svg
dark: ../docs/assets/skillevaluator-logo.svg
height: 28
right-text: SkillEvaluator
favicon: ../docs/assets/skillevaluator-logo.svg
navigation:
- page: Overview
path: ../docs/index.mdx
slug: /
- section: Get Started
skip-slug: true
contents:
- page: Installation
path: ../docs/installation.mdx
slug: installation
- page: Quickstart
path: ../docs/quickstart.mdx
slug: quickstart
- page: Repository Integration
path: ../docs/repository-integration.mdx
slug: repository-integration
- page: Evaluate from Your Agent
path: ../docs/agent-skill.mdx
slug: agent-skill
- section: Set Up
skip-slug: true
contents:
- page: Providers & Credentials
path: ../docs/configuration.mdx
slug: configuration
- page: Agents & Sandboxes
path: ../docs/agents-and-sandboxes.mdx
slug: agents-and-sandboxes
- section: Evaluate Your Skill
skip-slug: true
contents:
- page: "Tier 1: Validation"
path: ../docs/tier1-validation.mdx
slug: tier1-validation
- page: "Tier 1: Harness Check"
path: ../docs/harness-check.mdx
slug: harness-check
- page: "Tier 2: Deduplication"
path: ../docs/tier2-deduplication.mdx
slug: tier2-deduplication
- page: "Tier 3: Live Agent Evaluation"
path: ../docs/tier3-live-evaluation.mdx
slug: tier3-live-evaluation
- page: Prepare a Dataset
path: ../docs/prepare-dataset.mdx
slug: prepare-dataset
- page: Configure a Tier 3 Run
path: ../docs/tier3-run-options.mdx
slug: tier3-run-options
- page: NeMo Gym Integration
path: ../docs/nemo-gym.mdx
slug: nemo-gym
- page: Grader Validation
path: ../docs/grader-validation.mdx
slug: grader-validation
- page: Trace Datasets
path: ../docs/trace-datasets.mdx
slug: trace-datasets
- page: Custom Graders & Tasks
path: ../docs/custom-graders.mdx
slug: custom-graders
- page: Reproducible Terminal Demo
path: ../docs/starter-demo.mdx
slug: starter-demo
- section: Results & CI
skip-slug: true
contents:
- page: Reports & Results
path: ../docs/reports.mdx
slug: reports
- page: Metrics Reference
path: ../docs/metrics-reference.mdx
slug: metrics-reference
- page: Performance Dashboard
path: ../docs/performance-dashboard.mdx
slug: performance-dashboard
- page: "BENCHMARK.md Rollout and Backfill"
path: ../docs/benchmark-rollout.mdx
slug: benchmark-rollout
- page: GitHub Actions
path: ../docs/github-actions.mdx
slug: github-actions
- page: GitHub CLI Setup
path: ../docs/cli-onboarding.mdx
slug: cli-onboarding
- page: GitLab CI/CD
path: ../docs/gitlab-ci.mdx
slug: gitlab-ci
- page: Gate Your CI
path: ../docs/ci-integration.mdx
slug: ci-integration
- section: Reference
skip-slug: true
contents:
- page: Installation Options
path: ../docs/installation-options.mdx
slug: installation-options
- page: Tier 1 Checks and Policies
path: ../docs/tier1-reference.mdx
slug: tier1-reference
- page: Tier 2 Checks and Limits
path: ../docs/tier2-reference.mdx
slug: tier2-reference
- section: Live Evaluation
skip-slug: true
contents:
- page: Run Options and Cost
path: ../docs/tier3-run-reference.mdx
slug: tier3-run-reference
- page: Dataset Format and Policy
path: ../docs/eval-datasets.mdx
slug: eval-datasets
- page: Interpret Results
path: ../docs/tier3-results.mdx
slug: tier3-results
- page: Grading and Evidence
path: ../docs/tier3-grading.mdx
slug: tier3-grading
- page: Runtime and Backends
path: ../docs/tier3-runtime.mdx
slug: tier3-runtime
- page: Interaction and Activation
path: ../docs/tier3-experiments.mdx
slug: tier3-experiments
- page: CLI Reference
path: ../docs/cli-reference.mdx
slug: cli-reference
- page: Environment Variables
path: ../docs/environment-variables.mdx
slug: environment-variables
- page: Advanced CI
path: ../docs/advanced-ci.mdx
slug: advanced-ci
- page: Contributing
path: ../docs/developer-guide.mdx
slug: developer-guide
- link: NVIDIA Verified Skills
href: https://docs.nvidia.com/skills/