Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
Python agent-evaluationagent-pluginsagent-securityagent-skillsagentic-aibenchmarkci-templatesclaude-codecodexevaluateevaluationharnesspluginplugin-evaluationsecurity-scannerskill-evalskill-evaluationskill-evaluatorskill-gap-analysisskills
---
title: "GitLab CI/CD"
description: "Add quality and security checks to GitLab with no model key required."
---
Bring quality and security checks to your repository. No model key required.
Optional LLM checks are available. Point the reusable GitLab CI job at your skills
and choose a quality threshold; merge requests evaluate changed skills
without installing scanners or adding evaluation code to your repository.
The shared container includes SkillSpector security scanning.
## Get started
The shared image applies one default validation policy to public and private
repositories. No `internal`/`external` variable is needed. Older image reports
label the same policy `external`; repository access still controls scan artifacts.
Copy [gitlab-skill-checks.yml](https://github.com/NVIDIA/SkillEvaluator/blob/main/ci/bundles/1.0.1/gitlab-skill-checks.yml)
from the [ready-to-use templates](https://github.com/NVIDIA/SkillEvaluator/tree/main/ci/bundles/1.0.1).
It includes the pinned shared configuration and scanner image. You do not need
a GitLab mirror or a local onboarding CLI.
<Accordion title="I prefer to inspect the source template">
Browse the [GitLab source template](https://github.com/NVIDIA/SkillEvaluator/blob/main/ci/templates/gitlab-skill-checks.yml)
and [template catalog](https://github.com/NVIDIA/SkillEvaluator/blob/main/ci/templates/README.md).
Source files contain placeholders; use the filled-in copy when onboarding.
</Accordion>
<Steps>
### Add the reusable job
Copy `gitlab-skill-checks.yml` from the template folder using GitHub's **Raw** view. For a new pipeline,
use its contents as `.gitlab-ci.yml`. For an existing pipeline, merge its
`include`, variables and `skill-evaluation` rules into your root file.
### Choose your skills and score
Set `SKILL_EVAL_PATHS` to your skill directory and `SKILL_EVAL_MIN_SCORE` to the
required quality score from 0 to 100 (default 70). For multiple roots, use a
YAML multiline string.
### Open a merge request
The job evaluates changed skills, shows the result in the pipeline, and retains
reports as job artifacts, including when a check fails.
</Steps>
The generated file has this shape; the release fills in the immutable pins:
```yaml title=".gitlab-ci.yml"
include:
- remote: https://raw.githubusercontent.com/NVIDIA/SkillEvaluator/RELEASE_WORKFLOW_SHA/ci/gitlab/skill-evaluation.yml
variables:
SKILL_EVAL_IMAGE: ghcr.io/nvidia/skillevaluator-tier1@sha256:RELEASE_IMAGE_DIGEST
SKILL_EVAL_PATHS: "skills"
SKILL_EVAL_MIN_SCORE: "70"
skill-evaluation:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
- if: '$CI_PIPELINE_SOURCE == "web"'
```
A compatible runner must be able to pull the image and run Linux containers.
If it requires tags, add them to `skill-evaluation`. Static scanning needs no
model key, but dependency vulnerability checks may contact external services.
## See the results
Open the `skill-evaluation` job and download its `skill-evaluator-report`
artifact. Open `index.html` for the consolidated view, then select a skill for
its detailed HTML report. JSON, Markdown and SARIF are included.
<Frame caption="Real demo result: the skill scored 98.2 and failed a 100-point threshold. The score and the scan verdict are separate.">
<img src="./assets/repository-ci-results-example.png" alt="Example report table showing a failed skill with a quality score of 98.2 and a link to its report" />
</Frame>
The shared `skill-evaluation` job writes JSON, Markdown and SARIF reports; it
does not produce JUnit XML yet. To show findings as a native GitLab
merge-request test report, run SkillEvaluator in your own job with `-r junit`
and attach the file it writes. `tier1` writes
`reports/skillevaluator-tier1.junit.xml`:
```yaml title="Run it yourself: JUnit test report"
skill-junit:
script:
- skillevaluator tier1 skills/foo -r junit -o reports
artifacts:
when: always
reports:
junit: reports/skillevaluator-tier1.junit.xml
```
`skillevaluator validate` on a folder of skills writes one file per skill
instead, as `reports/<skill>/skillevaluator-output-<timestamp>.junit.xml`;
point `junit:` at `reports/**/*.junit.xml` to include nested skill folders.
The [public examples](https://github.com/NVIDIA/SkillEvaluator/blob/main/ci/examples/README.md)
include this report preview and a log excerpt without requiring access to a
separate demo project.
<Tabs>
<Tab title="Job log">
The log identifies the skill, result and quality score. This shortened
excerpt is from the real demo run shown above:
```text
Skill result: "skills/glossary-builder" — FAILED
Quality: 98.2/100, grade "A"
Check: "QUALITY" — "failed"
Scanner error: "Quality score 98.2/100 is below the required 100/100 ..."
```
</Tab>
<Tab title="Report artifact">
Download the artifact from the job to inspect the overview and per-skill
findings. Artifacts expire according to the job's retention settings.
</Tab>
</Tabs>
<AccordionGroup>
<Accordion title="Author emails and reviewed security exceptions">
Keep the `metadata.author` attribution. Extra email copies in
contact fields or documents are checked separately; use a handle or a
contact-directory link instead. See [author attribution](/skills/skillevaluator/tier1-reference#author-attribution-and-contact-emails).
For reviewed SkillSpector exceptions, set
`SKILL_EVAL_SECURITY_BASELINES: "true"` and add
`.skillevaluator/skillspector-baselines.yaml` using a template and image
version that include baseline support. CI reads the approved mapping
and baseline files from the merge request's target revision. Regenerate and
review fingerprint exceptions when the scanner version changes. Baselines
do not waive PII or incomplete analysis. Follow the [baseline setup guide](https://github.com/NVIDIA/SkillEvaluator/blob/main/ci/README.md#reviewed-security-baselines).
</Accordion>
<Accordion title="When does it run?">
`SKILL_EVAL_SCAN_MODE: auto` evaluates changed skills in detached
merge-request pipelines and all configured skills on the default branch or
a manual pipeline. Set it to `all` for a full scan each time. Changed-only
selection needs the merge-request diff base and full Git history; missing
history is an error, not a pass. Merged-results and merge-train pipelines
must use `all` until changed-only selection is supported for them. Shared
files outside skill directories do not select dependent skills; use `all`
when those changes require a complete scan.
</Accordion>
<Accordion title="Fit the job into an existing pipeline">
GitLab requires merge-request `rules` in the root `.gitlab-ci.yml`, even
when a job is included. The generated file provides them. If your project
has `workflow: rules`, ensure they allow merge-request pipelines while
preserving your existing policy. The job uses the `test` stage by default;
override its stage if your pipeline does not define it. A failed or
incomplete job fails the pipeline; configure your GitLab merge policy if it
should also block merging.
</Accordion>
<Accordion title="Scores, reports and privacy">
The threshold applies to the quality score; security, schema and other
required checks remain independent gates. The scanner runs quality and
security checks and saves reports as job artifacts; it does not host reports
or post merge-request comments automatically. Known values detected by
credential patterns, NVIDIA API
keys, and unquoted `KEY=value` and `Bearer` credentials are redacted,
including parts of them that a scanner cuts or escapes, but not every
secret or transformation is detectable. Raw scanner logs are
withheld when a credential is present. Remove and rotate exposed
credentials; previous artifacts and Git history are not rewritten.
</Accordion>
</AccordionGroup>
For GitHub and other CI systems, see [Repository Integration](/skills/skillevaluator/repository-integration).
<div id="three-tier-checks-and-feedback">
<div id="merge-into-an-existing-root-configuration">
For the explicit all-tier opt-in and its prerequisites, see [Advanced CI](/skills/skillevaluator/advanced-ci).
The [GitLab setup](https://github.com/NVIDIA/SkillEvaluator/blob/main/ci/templates/all-tiers/README.md#gitlab-checks-and-feedback) and [all-tier pipeline migration](https://github.com/NVIDIA/SkillEvaluator/blob/main/ci/templates/all-tiers/README.md#merge-into-an-existing-root-configuration) references preserve the detailed contracts.
</div>
</div>