Skip to content

Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.

docs: repair onboarding page navigation (#573)

Christopher Kevina0affcc9fe43
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f8ac0786..3917cd97 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -256,6 +256,9 @@ All notable changes to SkillEvaluator are documented in this file.
successful stop, instead of reporting an interrupted cleanup on cancellation.
- Native NeMo Gym cleanup now also stops run-owned detached server processes
after startup timeouts, interruptions and early exits, without targeting other runs.
+- Published documentation links use page routes so readers can navigate between
+ setup, report interpretation, recovery and the reference pages.
+
- NeMo Gym results reject verifier rewards that identify a different staged
case and retain custom grader explanations in collected results.
- The Copilot CLI `CP-NO-FRONTMATTER` finding and the plugin component findings
diff --git a/docs/agent-skill.mdx b/docs/agent-skill.mdx
index 18bc447e..b11676a6 100644
--- a/docs/agent-skill.mdx
+++ b/docs/agent-skill.mdx
@@ -48,7 +48,7 @@ pip install --upgrade skillevaluator # Tier 1 and 2
pip install --upgrade "skillevaluator[all]" # adds Tier 3 live evaluation
```
-See [Installation](installation.mdx) for extras and Python versions.
+See [Installation](/skills/skillevaluator/installation) for extras and Python versions.
### Add the skill to your agent
@@ -100,7 +100,7 @@ export NVIDIA_API_KEY=...
```
OpenAI, Anthropic, and any OpenAI-compatible gateway also work; see
-[Providers & Credentials](configuration.mdx). Keys stay in your environment; the
+[Providers & Credentials](/skills/skillevaluator/configuration). Keys stay in your environment; the
skill tells the agent never to print or store them.
</Steps>
@@ -161,7 +161,7 @@ since been removed.
<Accordion title="Tier 3: in a sandbox">
Docker is the default backend; SkillEvaluator drives Harbor for you. For trusted skills,
`--env-mode local` runs the agent on the host under Bubblewrap (Linux) or Seatbelt (macOS).
- See [Agents & Sandboxes](agents-and-sandboxes.mdx).
+ See [Agents & Sandboxes](/skills/skillevaluator/agents-and-sandboxes).
</Accordion>
<Accordion title="Incomplete runs">
If an attempt errors or times out, the run is INCOMPLETE and no final score or lift is
@@ -173,16 +173,16 @@ since been removed.
## Next steps
<CardGroup cols={2}>
- <Card title="Tier 3: Live Evaluation" icon="flask" href="tier3-live-evaluation.mdx">
+ <Card title="Tier 3: Live Evaluation" icon="flask" href="/skills/skillevaluator/tier3-live-evaluation">
How baseline and with-skill runs are scored, and what Skill Lift means.
</Card>
- <Card title="Eval Datasets" icon="list-check" href="eval-datasets.mdx">
+ <Card title="Eval Datasets" icon="list-check" href="/skills/skillevaluator/eval-datasets">
The `evals.json` contract the agent writes and validates.
</Card>
- <Card title="Reports & Results" icon="chart-column" href="reports.mdx">
+ <Card title="Reports & Results" icon="chart-column" href="/skills/skillevaluator/reports">
Every report file the agent reads, and what each field means.
</Card>
- <Card title="Bring It to Your Repository" icon="code-branch" href="repository-integration.mdx">
+ <Card title="Bring It to Your Repository" icon="code-branch" href="/skills/skillevaluator/repository-integration">
Check every skill change in GitHub or GitLab CI.
</Card>
</CardGroup>
diff --git a/docs/agents-and-sandboxes.mdx b/docs/agents-and-sandboxes.mdx
index 361deda7..6426f5eb 100644
--- a/docs/agents-and-sandboxes.mdx
+++ b/docs/agents-and-sandboxes.mdx
@@ -8,7 +8,7 @@ run: which agents attempt the tasks (`--agents`) and where the trials execute
(`--env-mode`). This page is the complete matrix for both, plus the resource,
safety, and workspace controls around them and the readiness checks that tell
you a run will work before you spend compute. The run workflow itself is
-[Tier 3: Live Evaluation](tier3-live-evaluation.mdx).
+[Tier 3: Live Evaluation](/skills/skillevaluator/tier3-live-evaluation).
[Harbor](https://github.com/harbor-framework/harbor) is the open-source agent
evaluation framework that executes each Tier 3 trial and provides its Docker,
@@ -147,7 +147,7 @@ refusal in 20 cases with 3 attempts each, the run's `result.json` records
does `validate --block-on-agent-eval`. On a route that serves the fallback
model, such as the Anthropic API, the retry used to finish such an attempt,
and the attempt was scored, so a run that completed before can now fail. See
-[Troubleshooting](tier3-live-evaluation.mdx#troubleshooting) for what to do.
+[Troubleshooting](/skills/skillevaluator/tier3-live-evaluation#troubleshooting) for what to do.
The `claude` alias is canonicalized to `claude-code` everywhere an agent
name can appear — `-a/--agents`, `--agent-model`, and `harbor.agents` in
@@ -166,7 +166,7 @@ skillevaluator tier3 evaluate ./my-skill --agents codex --env-mode docker \
```
To make an override persistent, set it in `evals/config.yml` under
-`harbor.agents.<name>.model` — see [Eval Datasets](eval-datasets.mdx).
+`harbor.agents.<name>.model` — see [Eval Datasets](/skills/skillevaluator/eval-datasets).
<Note>
Harbor installs agent CLIs in Docker and remote task environments. Local mode
@@ -176,7 +176,7 @@ Operator credentials come only from the host environment — a
credential name (`ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`,
`CLAUDE_CODE_USE_BEDROCK`, `NVIDIA_API_KEY`, `OPENAI_API_KEY`,
`OPENAI_BASE_URL`) or references one with `${...}` is rejected at run time.
-See the two credential roles in [Providers & Credentials](configuration.mdx).
+See the two credential roles in [Providers & Credentials](/skills/skillevaluator/configuration).
</Note>
## One NVIDIA Build key for everything
@@ -601,7 +601,7 @@ skillevaluator tier3 evaluate ./my-skill --agents codex --env-mode docker \
To make the sizing part of the skill's run policy instead of a flag, set
`harbor.resources` (`cpus`, `memory_mb`, `storage_mb`) in `evals/config.yml`
-— see [Eval Datasets](eval-datasets.mdx). A trial that dies mid-run with the
+— see [Eval Datasets](/skills/skillevaluator/eval-datasets). A trial that dies mid-run with the
agent process killed is the classic out-of-memory symptom; raise
`--override-memory-mb` first.
@@ -637,7 +637,7 @@ skips the check.
Environment rules — size limits, auto-generated fallbacks, and sidecar
containers — are part of the `evals/` contract in
-[Eval Datasets](eval-datasets.mdx).
+[Eval Datasets](/skills/skillevaluator/eval-datasets).
Docker Compose project `.env` files are rejected before startup. Protected
main-container and sidecar operations disable Compose's implicit env-file
@@ -746,7 +746,7 @@ The sandbox policy is controlled by environment variables:
| `SKILLEVALUATOR_RUNTIME_DIR` | `~/.local/share/skillevaluator/runtimes` | Where local mode looks for managed agent CLIs. Must be a dedicated subdirectory — not the home directory or one of its parents. |
Full semantics for each variable are in
-[Environment Variables](environment-variables.mdx). `doctor --env-mode local`
+[Environment Variables](/skills/skillevaluator/environment-variables). `doctor --env-mode local`
verifies the sandbox backend and the installed agent CLIs before you commit to
a run.
@@ -867,7 +867,7 @@ any check fails, so either can gate a script or CI job. Warnings from
inconclusive rather than a definitive rejection. Both commands check Tier 3 by
default. `--tier 1` checks the Tier 1 scanners, `--tier 2` the embedding and LLM
providers, and `--tier all` every tier; see the
-[CLI Reference](cli-reference.mdx#doctor).
+[CLI Reference](/skills/skillevaluator/cli-reference#doctor).
### doctor
@@ -925,16 +925,16 @@ skillevaluator harbor-view <jobs-dir>
The retained tree holds the full agent trajectory and grader output for every
trial — the fastest way to see what an agent actually did. The results layout
and where `<jobs-dir>` lands on disk are in
-[Reports & Results](reports.mdx); debugging custom graders this way is
-covered in [Custom Graders & Tasks](custom-graders.mdx).
+[Reports & Results](/skills/skillevaluator/reports); debugging custom graders this way is
+covered in [Custom Graders & Tasks](/skills/skillevaluator/custom-graders).
## Next steps
<CardGroup cols={2}>
- <Card title="Tier 3: Live Evaluation" icon="flask" href="tier3-live-evaluation.mdx">
+ <Card title="Tier 3: Live Evaluation" icon="flask" href="/skills/skillevaluator/tier3-live-evaluation">
The end-to-end run workflow — readiness, dataset, evaluation, and results.
</Card>
- <Card title="Custom Graders & Tasks" icon="terminal" href="custom-graders.mdx">
+ <Card title="Custom Graders & Tasks" icon="terminal" href="/skills/skillevaluator/custom-graders">
Bring your own grader or Harbor task when the defaults can't express
success.
</Card>
diff --git a/docs/cli-reference.mdx b/docs/cli-reference.mdx
index a1cb1fee..62a2ca12 100644
--- a/docs/cli-reference.mdx
+++ b/docs/cli-reference.mdx
@@ -154,7 +154,7 @@ errors, such as an option the workflow does not accept, exit `2`.
| `--quality-config PATH` | packaged `quality-default.yaml` | YAML file that adjusts the quality check's expected headings and error-handling rules; settings it omits keep their packaged values. See [Tier 1: Validation](/skills/skillevaluator/tier1-validation#the-checks). |
| `--pii-waivers PATH` | none | Reviewed, expiring PII waivers applied by the `pii` check. See [PII waivers](/skills/skillevaluator/tier1-validation#pii-waivers). |
| `--profile TEXT` | `SKILLEVALUATOR_PROFILE`, then `default` | Validation policy: `default` or `registry`. `external` is a compatibility alias for `default`. |
-| `--policy FILE` | none | Explicit policy overlay, including reviewed ownership-contact severity. See [Tier 1 policies](tier1-validation.mdx#validation-profiles-and-policies). |
+| `--policy FILE` | none | Explicit policy overlay, including reviewed ownership-contact severity. See [Tier 1 policies](/skills/skillevaluator/tier1-validation#validation-profiles-and-policies). |
| `--previous-version TEXT` | `SKILLEVALUATOR_PREVIOUS_VERSION`, else unset | Previous version for version checks |
| `--skillspector-baseline FILE` | unset | Audited SkillSpector suppression baseline; never discovered from the skill. See [Audited SkillSpector baselines](/skills/skillevaluator/tier1-validation#audited-skillspector-baselines). |
diff --git a/docs/configuration.mdx b/docs/configuration.mdx
index 2b65cad4..63ee8d4b 100644
--- a/docs/configuration.mdx
+++ b/docs/configuration.mdx
@@ -345,10 +345,10 @@ Setting `SKILLSPECTOR_PROVIDER` yourself overrides the bridge, but it must name
## Next steps
<CardGroup cols={2}>
- <Card title="Tier 2: Deduplication" icon="copy" href="tier2-deduplication.mdx">
+ <Card title="Tier 2: Deduplication" icon="copy" href="/skills/skillevaluator/tier2-deduplication">
Put your embedding provider to work — find overlap within and across skills.
</Card>
- <Card title="Tier 3: Live Evaluation" icon="flask" href="tier3-live-evaluation.mdx">
+ <Card title="Tier 3: Live Evaluation" icon="flask" href="/skills/skillevaluator/tier3-live-evaluation">
Run a live agent evaluation with the two-credential setup above.
</Card>
</CardGroup>
diff --git a/docs/custom-graders.mdx b/docs/custom-graders.mdx
index 28f78008..161f0a41 100644
--- a/docs/custom-graders.mdx
+++ b/docs/custom-graders.mdx
@@ -10,7 +10,7 @@ your own complete [Harbor](https://github.com/harbor-framework/harbor) task.
Harbor is the open-source agent evaluation framework that executes Tier 3
trials. Scaffolding is keyless and takes one command; running a custom-graded
evaluation needs the same Tier 3 stack as any live eval — see
-[Tier 3: Live Evaluation](tier3-live-evaluation.mdx).
+[Tier 3: Live Evaluation](/skills/skillevaluator/tier3-live-evaluation).
## Grading modes
@@ -88,7 +88,7 @@ optional — the config already selects it.
`custom_only` still requires a numeric overall score. Qualitative reasons can
accompany it, but prose alone is not a reward. For a manual, evidence-linked
assessment alongside current scores, see the
-[qualitative assessment review contract](reports.mdx#qualitative-assessment-review-contract).
+[qualitative assessment review contract](/skills/skillevaluator/reports#qualitative-assessment-review-contract).
Its worksheet is a documentation prototype, not input to your grader or a
scoreless runtime mode.
@@ -172,7 +172,7 @@ Role log directories retain their host locations, so use
deriving a parent from a role directory or writing `../artifacts` from it.
Your grader reads Harbor's `trajectory.json` as written. Unlike the standard
-grader, it does not [rebuild a missing or empty trajectory](tier3-live-evaluation.mdx#the-graded-trajectory)
+grader, it does not [rebuild a missing or empty trajectory](/skills/skillevaluator/tier3-live-evaluation#the-graded-trajectory)
from agent logs, so handle that case explicitly. A step's `message` and an
observation result's `content` can each be a string or a list of content parts
(`text`, `image`, or `audio`), so read both forms.
@@ -196,7 +196,7 @@ skillevaluator init-harbor-task ./my-skill
| `--force` | off | Overwrite an existing starter case |
Exact defaults for both scaffolding commands are in the
-[CLI Reference](cli-reference.mdx#init-custom-grader).
+[CLI Reference](/skills/skillevaluator/cli-reference#init-custom-grader).
The scaffold produces a complete, runnable task source:
@@ -235,7 +235,7 @@ Two rules keep BYOT predictable:
`[metadata]` in `task.toml`, falling back to the case directory name — must
match an entry `id` in `evals/evals.json`, or the run refuses to start.
`custom_only` has no such requirement. See
- [Eval Datasets](eval-datasets.mdx).
+ [Eval Datasets](/skills/skillevaluator/eval-datasets).
- **Your source is never mutated in place.** SkillEvaluator stages with-skill
and baseline copies of `evals/harbor/` under the results directory for each
run; the files you author stay untouched. In a paired run, the staged copies
@@ -310,7 +310,7 @@ skillevaluator harbor-view <jobs-dir>
Grader stdout, the written `reward.json`, and the full trajectory are all in
the retained job tree — the fastest way to find out why a score isn't what
-you expected. [Agents & Sandboxes](agents-and-sandboxes.mdx) covers the
+you expected. [Agents & Sandboxes](/skills/skillevaluator/agents-and-sandboxes) covers the
debugging workflow in depth.
## What reports show
@@ -334,7 +334,7 @@ Custom scores flow into the same results tree as everything else:
run's attempt policy.
Custom lift needs both arms — a `--skip-baseline` run reports your custom
-scores but no lift. [Reports & Results](reports.mdx) covers the full on-disk
+scores but no lift. [Reports & Results](/skills/skillevaluator/reports) covers the full on-disk
layout and the machine-readable contract.
## Import precomputed custom metrics
@@ -409,7 +409,7 @@ must have a nonempty `custom_metrics` object, but its declared metrics may all
be null. Arm means exclude missing values and report a scored count per metric
against total observations. All-missing metrics render `NO SCORE`.
-Case IDs are strings following the [dataset identity rules](eval-datasets.mdx). Attempts are
+Case IDs are strings following the [dataset identity rules](/skills/skillevaluator/eval-datasets). Attempts are
integers from 1 through 1000, not booleans. Only `with_skill` and `without_skill`
arms are accepted; duplicate case/attempt/arm identities and case IDs differing
only in capitalization are rejected. Standard metric names, metadata names
@@ -480,11 +480,11 @@ scores produce null delta/improvement and `NO SCORE`, not a fabricated zero.
## Next steps
<CardGroup cols={2}>
- <Card title="Eval Datasets" icon="book" href="eval-datasets.mdx">
+ <Card title="Eval Datasets" icon="book" href="/skills/skillevaluator/eval-datasets">
The full evals/ contract — dataset format, config.yml run policy, and
fixtures your custom task can build on.
</Card>
- <Card title="Reports & Results" icon="terminal" href="reports.mdx">
+ <Card title="Reports & Results" icon="terminal" href="/skills/skillevaluator/reports">
Everything a run writes to disk and how to read it, including where
custom metrics and custom lift surface.
</Card>
diff --git a/docs/developer-guide.mdx b/docs/developer-guide.mdx
index 6cef57e0..b29ef1b7 100644
--- a/docs/developer-guide.mdx
+++ b/docs/developer-guide.mdx
@@ -93,7 +93,7 @@ The script renders its own fixture reports, then writes screenshots and `navigat
## Documentation and starter changes
-The [terminal starter](starter-demo.mdx) uses a public skill, task and input with
+The [terminal starter](/skills/skillevaluator/starter-demo) uses a public skill, task and input with
real CLI output checks. For changes to it, run:
```bash
@@ -166,7 +166,7 @@ Contributions flow through GitHub pull requests:
2. **Keep each pull request focused.** Use the pull request template and explain the user-visible behavior and the verification you performed.
3. **Add focused tests** for behavioral changes.
4. **Update [CHANGELOG.md](https://github.com/NVIDIA/SkillEvaluator/blob/main/CHANGELOG.md)** when the change affects users.
-5. **Keep provider credentials, customer data, and private benchmark material out of the repository.** Tier 3 live evaluation reads provider and agent credentials from your environment, never from the repo (with NVIDIA Build, one `NVIDIA_API_KEY` covers both roles) — see [Providers & Credentials](configuration.mdx).
+5. **Keep provider credentials, customer data, and private benchmark material out of the repository.** Tier 3 live evaluation reads provider and agent credentials from your environment, never from the repo (with NVIDIA Build, one `NVIDIA_API_KEY` covers both roles) — see [Providers & Credentials](/skills/skillevaluator/configuration).
<Warning>
Every commit needs a Developer Certificate of Origin sign-off — commit with `git commit -s`. Pull requests containing unsigned commits are not accepted. By contributing, you confirm you have the right to submit the work under the Apache License 2.0.
diff --git a/docs/environment-variables.mdx b/docs/environment-variables.mdx
index 3c3e1db1..9c79373d 100644
--- a/docs/environment-variables.mdx
+++ b/docs/environment-variables.mdx
@@ -21,7 +21,7 @@ contract. None belongs to either SkillEvaluator prefix.
## LLM provider
-These variables select and configure the provider used for LLM-backed checks and judging. The setup walkthrough, including per-provider export blocks and default models, is in [Providers & Credentials](configuration.mdx).
+These variables select and configure the provider used for LLM-backed checks and judging. The setup walkthrough, including per-provider export blocks and default models, is in [Providers & Credentials](/skills/skillevaluator/configuration).
| Variable | Default | Effect |
| --- | --- | --- |
@@ -116,7 +116,7 @@ The `--llm` security analysis runs [SkillSpector](https://github.com/NVIDIA/Skil
## Embeddings
-Tier 2 semantic checks use an OpenAI-compatible embeddings API. Workflow context lives in [Providers & Credentials](configuration.mdx#embeddings) and [Tier 2: Deduplication](tier2-deduplication.mdx).
+Tier 2 semantic checks use an OpenAI-compatible embeddings API. Workflow context lives in [Providers & Credentials](/skills/skillevaluator/configuration#embeddings) and [Tier 2: Deduplication](/skills/skillevaluator/tier2-deduplication).
| Variable | Default | Effect |
| --- | --- | --- |
@@ -127,7 +127,7 @@ Tier 2 semantic checks use an OpenAI-compatible embeddings API. Workflow context
## Validation (Tier 1)
-These variables tune the static and security checks. The check-by-check guide is [Tier 1: Validation](tier1-validation.mdx).
+These variables tune the static and security checks. The check-by-check guide is [Tier 1: Validation](/skills/skillevaluator/tier1-validation).
| Variable | Default | Effect |
| --- | --- | --- |
@@ -148,9 +148,9 @@ The scanner path overrides fail closed: a value that is not an absolute, executa
## Live evaluation (Tier 3)
These variables control where results land and how Harbor confines local mode.
-The run workflow is [Tier 3: Live Evaluation](tier3-live-evaluation.mdx);
+The run workflow is [Tier 3: Live Evaluation](/skills/skillevaluator/tier3-live-evaluation);
execution environments are covered in
-[Agents & Sandboxes](agents-and-sandboxes.mdx).
+[Agents & Sandboxes](/skills/skillevaluator/agents-and-sandboxes).
| Variable | Default | Effect |
| --- | --- | --- |
@@ -167,15 +167,15 @@ execution environments are covered in
| `SKILLEVALUATOR_LOCAL_STRICT_READS` | `false` | Tightens the sandbox's read-only view of the host system to a stricter path set. |
| `SKILLEVALUATOR_LOCAL_INHERIT_AGENT_KEYS` | `false` | When true, local-mode trials inherit the host's agent credential variables into their ambient environment. Off by default so a hostile skill command cannot read credentials — agents and verifiers receive credentials per-exec instead. |
| `SKILLEVALUATOR_RUNTIME_DIR` | `~/.local/share/skillevaluator/runtimes` | Where local mode looks for managed agent CLIs. Must be a dedicated subdirectory — not the home directory or one of its parents. |
-| `SKILLEVALUATOR_DISABLE_CUSTOM_BACKENDS` | unset (custom backends enabled) | Set to `1`, `true`, or `yes` (case-insensitive, with surrounding whitespace ignored) to hide every installed custom execution backend and reject its mode. Other values leave discovery enabled. Built-in modes remain available. See [Custom execution backends](agents-and-sandboxes.mdx#custom-execution-backends). |
+| `SKILLEVALUATOR_DISABLE_CUSTOM_BACKENDS` | unset (custom backends enabled) | Set to `1`, `true`, or `yes` (case-insensitive, with surrounding whitespace ignored) to hide every installed custom execution backend and reject its mode. Other values leave discovery enabled. Built-in modes remain available. See [Custom execution backends](/skills/skillevaluator/agents-and-sandboxes#custom-execution-backends). |
| `SKILLEVALUATOR_ALLOWED_MCP_HOSTS` | unset | Comma-separated hostnames approved for MCP endpoints receiving environment-variable references or authentication headers from skill-authored MCP declarations. Matching uses the whole hostname, case-insensitively; supply hostnames, not URLs or wildcard patterns. |
| `SKILLEVALUATOR_ALLOWED_MCP_SECRETS` | unset | Comma-separated, case-sensitive environment-variable names approved for references such as `${DEDICATED_MCP_TOKEN}` in skill-authored MCP declarations and `harbor.runtime_env`. Export the dedicated non-LLM credential on the host; literal secret values remain prohibited. Remote declarations also require the endpoint in `SKILLEVALUATOR_ALLOWED_MCP_HOSTS`. This allowlist cannot authorize operator-owned agent or provider credentials. |
| `SKILL_EVAL_HARBOR_PREBUILD_TASK_ENVS` | unset | Set to exactly `1` to opt in to pre-building Harbor task environment images before trials start (other truthy spellings are not honored). Opt-in because concurrent trial cleanup can remove a shared prebuilt image that sibling trials still need. |
| `SKILL_EVAL_JUDGE_MODEL` | falls back to `SKILL_EVAL_LLM_MODEL` | Operator-controlled model override for the Tier 3 dimension judge in standard grading. `LLM_JUDGE_MODEL` is honored as a higher-precedence spelling; when neither is set, the judge uses `SKILL_EVAL_LLM_MODEL`, then the built-in judge default. Both spellings change only the model used with the configured `SKILL_EVAL_LLM_PROVIDER`; they do not select a separate judge provider, so the model must be supported by that provider. Standard grading makes both spellings resolve to the selected override in Harbor's parent process, then forwards them through its verifier-only job layer, which takes precedence over task and step verifier declarations. Neither name is injected into the evaluated agent's initial environment; attempts to set or reference one from skill-authored `runtime_env` or native `[environment.env]` tables are rejected. Native custom-only graders keep ownership of their verifier configuration. Shared agent/verifier containers are a process boundary, not a confidentiality boundary; use a separate verifier environment when the model name itself must be hidden from untrusted agent processes. `run_config.json` records the configured judge provider, model, source, `override_applied`, reasoning effort, and catalog-verification status separately from agent models; an allowed provider fallback may use a different model for an individual judge call. When `override_applied` is false for a native Harbor task, `effective_model_source` is `native_harbor_runtime` and catalog verification is explicitly inconclusive because an authored task or step verifier declaration may supersede the recorded fallback. |
| `LLM_JUDGE_FALLBACK_MODELS` | unset | Comma-separated fallback models for the Tier 3 standard judges in the Harbor verifier. When the provider rejects the judge model as not found, invalid, or not accessible to the key, the verifier tries each fallback in order with the same provider. Fallback applies to the `openai`, `nv_build`, and `openai-compatible` providers. A safety refusal or an empty reply from the judge model also moves to the next fallback, on every provider. The value is forwarded only to standard verifier jobs, never to the agent environment or `custom_only` runs. |
| `SKILL_EVAL_LLM_REFUSAL_FALLBACK_MODEL` | `claude-opus-4-8` | Model named as the server-side refusal fallback on requests to Claude 5.5 or later models on the native Claude API; on Bedrock, the Tier 3 judge's refusal retry uses it. On Bedrock, a bare name such as `claude-opus-4-8` gets the judge model's geography prefix (`us.anthropic.`, `global.anthropic.`), and a full Bedrock ID such as `global.anthropic.claude-opus-4-8` is used as given. Set `none`, `off`, `disabled`, `false` or `0` to send no fallback, for example when the API answers HTTP 400 `is not a valid fallback target` for your model. An empty value counts as unset, and a fallback equal to the request model is never sent. Forwarded to standard verifier jobs like `LLM_JUDGE_FALLBACK_MODELS`. The reusable SkillEvaluator CI workflows do not take this variable; their provider-backed jobs keep the default. |
-| `LLM_JUDGE_FALLBACK_POLICY` | `append` | Opt-in limit on which models may answer a Tier 3 standard judge. `append` keeps the built-in refusal fallback, which sends a refused Claude 5.5 or later judge to `SKILL_EVAL_LLM_REFUSAL_FALLBACK_MODEL` (default `claude-opus-4-8`, off with `none`, never sent when it equals the judge model). `replace` turns it off, so only the judge model and the `LLM_JUDGE_FALLBACK_MODELS` entries answer. Values are case-insensitive; any other value stops a run with standard grading before trials start, and the verifier fails every standard judge on it before a request is sent. Forwarded like `LLM_JUDGE_FALLBACK_MODELS`, and recorded as `judge.fallback_policy` in `run_config.json` when set. See [Judge fallback and retry policies](tier3-live-evaluation.mdx#judge-fallback-and-retry-policies). |
-| `LLM_JUDGE_RETRY_POLICY` | `legacy` | Opt-in retry policy for the Tier 3 standard judges. `strict` tries at most three models (the judge model, then the first two `LLM_JUDGE_FALLBACK_MODELS` entries) with at most two requests each within the per-judge time budget, moves on only when a model is unavailable, ends the judge on a refusal or a quota, budget, safety, or security error, and scores only a first completed reply that is exactly one JSON object. `legacy` keeps the standard retries, fallbacks, format retry, and salvage. Values are case-insensitive; any other value stops a run with standard grading before trials start, and the verifier fails every standard judge on it before a request is sent. Forwarded like `LLM_JUDGE_FALLBACK_MODELS`, and recorded as `judge.retry_policy` in `run_config.json` when set. See [Judge fallback and retry policies](tier3-live-evaluation.mdx#judge-fallback-and-retry-policies). |
+| `LLM_JUDGE_FALLBACK_POLICY` | `append` | Opt-in limit on which models may answer a Tier 3 standard judge. `append` keeps the built-in refusal fallback, which sends a refused Claude 5.5 or later judge to `SKILL_EVAL_LLM_REFUSAL_FALLBACK_MODEL` (default `claude-opus-4-8`, off with `none`, never sent when it equals the judge model). `replace` turns it off, so only the judge model and the `LLM_JUDGE_FALLBACK_MODELS` entries answer. Values are case-insensitive; any other value stops a run with standard grading before trials start, and the verifier fails every standard judge on it before a request is sent. Forwarded like `LLM_JUDGE_FALLBACK_MODELS`, and recorded as `judge.fallback_policy` in `run_config.json` when set. See [Judge fallback and retry policies](/skills/skillevaluator/tier3-live-evaluation#judge-fallback-and-retry-policies). |
+| `LLM_JUDGE_RETRY_POLICY` | `legacy` | Opt-in retry policy for the Tier 3 standard judges. `strict` tries at most three models (the judge model, then the first two `LLM_JUDGE_FALLBACK_MODELS` entries) with at most two requests each within the per-judge time budget, moves on only when a model is unavailable, ends the judge on a refusal or a quota, budget, safety, or security error, and scores only a first completed reply that is exactly one JSON object. `legacy` keeps the standard retries, fallbacks, format retry, and salvage. Values are case-insensitive; any other value stops a run with standard grading before trials start, and the verifier fails every standard judge on it before a request is sent. Forwarded like `LLM_JUDGE_FALLBACK_MODELS`, and recorded as `judge.retry_policy` in `run_config.json` when set. See [Judge fallback and retry policies](/skills/skillevaluator/tier3-live-evaluation#judge-fallback-and-retry-policies). |
| `SKILLEVALUATOR_HARBOR_DIAGNOSTIC_ARTIFACT_MAX_BYTES` | `5242880` (5 MiB) | Size cap per diagnostic artifact copied from a Harbor trial into the results directory. Values are bounded to 0–67108864 bytes (64 MiB); `0` skips non-empty artifacts. Oversized artifacts are skipped and recorded in the artifact manifest. |
| `SKILLEVALUATOR_GKE_CLUSTER`, `SKILLEVALUATOR_GKE_REGION`, `SKILLEVALUATOR_GKE_NAMESPACE`, `SKILLEVALUATOR_GKE_REGISTRY_LOCATION`, `SKILLEVALUATOR_GKE_REGISTRY_NAME` | — | Target cluster, region, namespace, and Artifact Registry coordinates for `--env-mode gke` when not supplied via `--ek`. |
| `SKILLEVALUATOR_GKE_ALLOW_WORKLOAD_IDENTITY` | `false` | Opt in (`1`, `true`, `yes`) to single-pod GKE Workload Identity (`CLAUDE_CODE_USE_VERTEX=1` or a GCP-bound Kubernetes ServiceAccount). Restricted to trusted skills because the pod ServiceAccount and GCE/GKE metadata server are shared with evaluated skill commands. |
@@ -194,7 +194,7 @@ directory with `.env` loading and Harbor telemetry disabled:
- An evaluation run passes Harbor only an allowlisted environment: basic
process, locale, and temporary-directory variables such as `PATH`, `HOME`,
`LANG`, and `TMPDIR`; proxy and CA-bundle variables; the selected backend's
- own variables (see [Agents & Sandboxes](agents-and-sandboxes.mdx)); the
+ own variables (see [Agents & Sandboxes](/skills/skillevaluator/agents-and-sandboxes)); the
evaluator provider's settings and credential, plus the `AWS_*` chain for
`bedrock`; each selected agent's credentials; validated `runtime_env`
values; and the Tier 3 judge, retry, and evidence-budget variables on this
@@ -206,7 +206,7 @@ directory with `.env` loading and Harbor telemetry disabled:
so Docker Desktop can discover Compose. `PYTHONUTF8` and
`PYTHONIOENCODING` are also forwarded when set; use `PYTHONUTF8=1` and
`PYTHONIOENCODING=utf-8` for a non-UTF-8 locale. See the
- [PowerShell setup](agents-and-sandboxes.mdx#windows-hosts).
+ [PowerShell setup](/skills/skillevaluator/agents-and-sandboxes#windows-hosts).
- Setting `HARBOR_TELEMETRY` or `PYTHON_DOTENV_DISABLED` yourself has no
effect; SkillEvaluator always overrides both.
- Tier 3 needs python-dotenv 1.2.0 or newer, which the `tier3` extra installs.
@@ -220,7 +220,7 @@ to standard Tier 3 judge prompts. It is off by default; `true`, `yes`, and `on`
also enable it (case-insensitive). This operator-owned setting is forwarded only
to standard verifier jobs, never to agent environments or `custom_only` jobs.
When configured, its value is recorded as `judge.character_facts` in
-`run_config.json`. See [character-sensitive judging](tier3-live-evaluation.mdx#character-sensitive-judging)
+`run_config.json`. See [character-sensitive judging](/skills/skillevaluator/tier3-live-evaluation#character-sensitive-judging)
for scope, limits, and the effect on the goal-accuracy route.
Before the Harbor verifier calls the `accuracy`, `goal_accuracy`, and
@@ -241,7 +241,7 @@ Setting `SKILL_EVAL_BEHAVIOR_CHECK_BUDGET` below `SKILL_EVAL_BEHAVIOR_FINAL_RESP
plus 1600 is honoured, but logs a warning: the final response will not fit
beside any tool history, and `behavior_check` then reports incomplete evidence
instead of a score (see
-[behavior evidence completeness](tier3-live-evaluation.mdx#evidence-budgets)).
+[behavior evidence completeness](/skills/skillevaluator/tier3-live-evaluation#evidence-budgets)).
Lower the final-response limit alongside the budget to cap the judge prompt
without losing the answer.
@@ -262,7 +262,7 @@ In standard grading, a required judge transport, authentication, timeout, HTTP,
## In-container grader contract
-The `HARBOR_*` variables are read by your custom grader *inside* the task container — you never set them yourself, and each has a fixed in-container default. The full reward contract, with defaults and examples, lives in [Custom Graders & Tasks](custom-graders.mdx).
+The `HARBOR_*` variables are read by your custom grader *inside* the task container — you never set them yourself, and each has a fixed in-container default. The full reward contract, with defaults and examples, lives in [Custom Graders & Tasks](/skills/skillevaluator/custom-graders).
| Variable | What it carries |
| --- | --- |
@@ -277,14 +277,14 @@ The `HARBOR_*` variables are read by your custom grader *inside* the task contai
## Set at runtime, not committed
<Warning>
-Keep credentials out of skill repositories and CI configuration files. Export provider keys as masked secrets at runtime. Agent credentials for Tier 3 come only from the host environment — `evals/config.yml` `runtime_env` carries non-credential values, and any entry that names or references an operator-owned credential (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `NVIDIA_API_KEY`, their base-URL variables, `CLAUDE_CODE_USE_BEDROCK`, and the `AWS_*` credential variables) is rejected at run time, as are names that control the host process (`PATH`, `DOCKER_*`, proxy variables, `SKILL_EVAL_*`, `SKILLEVALUATOR_*`, and similar). See the blocked-names table in [Providers & Credentials](configuration.mdx#tier-3-credentials-two-roles). With NVIDIA Build, one exported `NVIDIA_API_KEY` covers both the evaluator and the agents in the `docker`, `kata`, and `local` env-modes.
+Keep credentials out of skill repositories and CI configuration files. Export provider keys as masked secrets at runtime. Agent credentials for Tier 3 come only from the host environment — `evals/config.yml` `runtime_env` carries non-credential values, and any entry that names or references an operator-owned credential (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `NVIDIA_API_KEY`, their base-URL variables, `CLAUDE_CODE_USE_BEDROCK`, and the `AWS_*` credential variables) is rejected at run time, as are names that control the host process (`PATH`, `DOCKER_*`, proxy variables, `SKILL_EVAL_*`, `SKILLEVALUATOR_*`, and similar). See the blocked-names table in [Providers & Credentials](/skills/skillevaluator/configuration#tier-3-credentials-two-roles). With NVIDIA Build, one exported `NVIDIA_API_KEY` covers both the evaluator and the agents in the `docker`, `kata`, and `local` env-modes.
</Warning>
## See also
-- [Providers & Credentials](configuration.mdx) — the setup walkthrough behind the provider variables
-- [CLI Reference](cli-reference.mdx) — the per-run flags that override many of these variables
-- [Gate Your CI](ci-integration.mdx) — supplying these variables as masked CI secrets
+- [Providers & Credentials](/skills/skillevaluator/configuration) — the setup walkthrough behind the provider variables
+- [CLI Reference](/skills/skillevaluator/cli-reference) — the per-run flags that override many of these variables
+- [Gate Your CI](/skills/skillevaluator/ci-integration) — supplying these variables as masked CI secrets
## GitLab all-tier admission
diff --git a/docs/eval-datasets.mdx b/docs/eval-datasets.mdx
index 61eae6a3..99e857b2 100644
--- a/docs/eval-datasets.mdx
+++ b/docs/eval-datasets.mdx
@@ -685,11 +685,11 @@ Passing checks and warnings are not listed as source errors.
## Next steps
<CardGroup cols={2}>
- <Card title="Run a live evaluation" icon="rocket" href="tier3-live-evaluation.mdx">
+ <Card title="Run a live evaluation" icon="rocket" href="/skills/skillevaluator/tier3-live-evaluation">
Point `tier3 evaluate` at your skill and measure Skill Lift with the
dataset you just authored.
</Card>
- <Card title="Custom graders and tasks" icon="flask" href="custom-graders.mdx">
+ <Card title="Custom graders and tasks" icon="flask" href="/skills/skillevaluator/custom-graders">
When assertions aren't enough, bring your own grader or a complete Harbor
task.
</Card>
diff --git a/docs/index.mdx b/docs/index.mdx
index 075d42b7..eace890a 100644
--- a/docs/index.mdx
+++ b/docs/index.mdx
@@ -6,7 +6,7 @@ layout: overview
SkillEvaluator is an open-source, multi-tier framework for evaluating AI agent artifacts, starting with agent skills: deterministic quality gates, semantic overlap detection, synthetic eval dataset generation, and live agent evaluation.
-Agent skills are folders of instructions and supporting files that extend AI agents, as defined by the [Agent Skills specification](https://agentskills.io/). SkillEvaluator is part of the [NVIDIA Verified Skills pipeline](https://docs.nvidia.com/skills/), published from the [NVIDIA/skills catalog](https://github.com/NVIDIA/skills). Tier 1 integrates [SkillSpector](https://github.com/NVIDIA/SkillSpector) for specialized security scanning; see [Installation](installation.mdx#system-tools) for the security setup.
+Agent skills are folders of instructions and supporting files that extend AI agents, as defined by the [Agent Skills specification](https://agentskills.io/). SkillEvaluator is part of the [NVIDIA Verified Skills pipeline](https://docs.nvidia.com/skills/), published from the [NVIDIA/skills catalog](https://github.com/NVIDIA/skills). Tier 1 integrates [SkillSpector](https://github.com/NVIDIA/SkillSpector) for specialized security scanning; see [Installation](/skills/skillevaluator/installation#system-tools) for the security setup.
<Note>
SkillEvaluator's support level is **Experimental**: community-supported on a best-effort basis through [GitHub Issues](https://github.com/NVIDIA/SkillEvaluator/issues), with no SLA. See [SUPPORT.md](https://github.com/NVIDIA/SkillEvaluator/blob/main/SUPPORT.md).
@@ -19,7 +19,7 @@ Agent skills are folders of instructions and supporting files that extend AI age
Choose a deterministic quality check or an LLM-backed rubric review. Both routes
use the bundled `text-analyzer` skill, so you can run them before preparing your
own skill. Neither route needs Docker or an agent runtime. Set up your chosen
-agent and sandbox when you move on to [live Tier 3 evaluation](tier3-live-evaluation.mdx).
+agent and sandbox when you move on to [live Tier 3 evaluation](/skills/skillevaluator/tier3-live-evaluation).
### Run without an LLM
@@ -35,11 +35,11 @@ uv run --python 3.13 skillevaluator quality-check src/skillevaluator/tier3/refer
Read the score, grade, and findings in the terminal, and open
`reports/first-quality/skillevaluator-quality.json` for the saved result.
This checks documentation quality; measuring the skill's effect on an agent
-requires a [live evaluation](tier3-live-evaluation.mdx).
+requires a [live evaluation](/skills/skillevaluator/tier3-live-evaluation).
-The [keyless Quickstart](quickstart.mdx#run-without-an-llm) explains the result
+The [keyless Quickstart](/skills/skillevaluator/quickstart#run-without-an-llm) explains the result
and how to check your own skill. The full static security suite has additional
-[scanner requirements](installation.mdx#system-tools), even though it also
+[scanner requirements](/skills/skillevaluator/installation#system-tools), even though it also
runs without an LLM or API key.
### Use NVIDIA Build
@@ -54,7 +54,7 @@ uv run --python 3.13 --extra llm skillevaluator rubric-eval src/skillevaluator/t
```
The `llm` extra installs the provider clients for this model call. The
-[NVIDIA Build Quickstart](quickstart.mdx#use-nvidia-build) covers access,
+[NVIDIA Build Quickstart](/skills/skillevaluator/quickstart#use-nvidia-build) covers access,
outputs, and the distinction between rubric scores and observed agent behavior.
## The three tiers
@@ -62,13 +62,13 @@ outputs, and the distinction between rubric scores and observed agent behavior.
Each tier answers one question about your skill.
<CardGroup cols={3}>
- <Card title="Tier 1: Validation" href="tier1-validation.mdx">
+ <Card title="Tier 1: Validation" href="/skills/skillevaluator/tier1-validation">
**Is this skill safe and well-formed?** Deterministic schema, quality, security, PII, license, and script checks that run offline — plus optional LLM-as-judge scoring.
</Card>
- <Card title="Tier 2: Deduplication" href="tier2-deduplication.mdx">
+ <Card title="Tier 2: Deduplication" href="/skills/skillevaluator/tier2-deduplication">
**Does it overlap with what exists?** Embedding similarity finds repeated guidance inside one skill and overlapping skills across a collection or catalog.
</Card>
- <Card title="Tier 3: Live Evaluation" href="tier3-live-evaluation.mdx">
+ <Card title="Tier 3: Live Evaluation" href="/skills/skillevaluator/tier3-live-evaluation">
**Does it actually help the agent?** A real agent runs generated tasks with and without the skill, inside a sandbox, and the difference is measured.
</Card>
</CardGroup>
@@ -77,9 +77,9 @@ Every tier is an independent entry point — run any command directly; nothing r
| Tier | Representative commands | Requires |
| --- | --- | --- |
-| [Tier 1](tier1-validation.mdx) | `tier1 PATH`, `quality-check`, `security-scan`, `pii-scan`, `lint-scripts`, `rubric-eval` | No API key for deterministic checks; external Semgrep, SkillSpector, and Gitleaks executables for full scanner coverage; a provider key for LLM checks |
-| [Tier 2](tier2-deduplication.mdx) | `context-optimization-check`, `similarity-check` | An embeddings provider; intra-skill analysis also needs a chat LLM — local OpenAI-compatible endpoints work |
-| [Tier 3](tier3-live-evaluation.mdx) | `create-eval-dataset`, `tier3 evaluate`, `compare` | No credential for keyless templates and report inspection; a provider key for LLM generation and grading; live evaluation also needs the agent CLI with its credential and a Docker, local OS, or cloud sandbox |
+| [Tier 1](/skills/skillevaluator/tier1-validation) | `tier1 PATH`, `quality-check`, `security-scan`, `pii-scan`, `lint-scripts`, `rubric-eval` | No API key for deterministic checks; external Semgrep, SkillSpector, and Gitleaks executables for full scanner coverage; a provider key for LLM checks |
+| [Tier 2](/skills/skillevaluator/tier2-deduplication) | `context-optimization-check`, `similarity-check` | An embeddings provider; intra-skill analysis also needs a chat LLM — local OpenAI-compatible endpoints work |
+| [Tier 3](/skills/skillevaluator/tier3-live-evaluation) | `create-eval-dataset`, `tier3 evaluate`, `compare` | No credential for keyless templates and report inspection; a provider key for LLM generation and grading; live evaluation also needs the agent CLI with its credential and a Docker, local OS, or cloud sandbox |
## How a run flows
@@ -89,7 +89,7 @@ Every tier is an independent entry point — run any command directly; nothing r
`skillevaluator validate ./my-skill` runs all three tiers by default, with
Tier 3 autopilot reusing an existing evaluation source or creating one starter
-case when none exists. Tier 1's deterministic gates run offline. Tier 2 and the optional Tier 1 LLM checks need a provider key, and Tier 3 adds the agent credential and a sandbox. Every run ends in [reports](reports.mdx) — CLI, JSON, HTML, or Markdown.
+case when none exists. Tier 1's deterministic gates run offline. Tier 2 and the optional Tier 1 LLM checks need a provider key, and Tier 3 adds the agent credential and a sandbox. Every run ends in [reports](/skills/skillevaluator/reports) — CLI, JSON, HTML, or Markdown.
## What Tier 3 measures
@@ -97,24 +97,24 @@ case when none exists. Tier 1's deterministic gates run offline. Tier 2 and the
- **Skill Lift** — the measured difference between the with-skill and without-skill runs; the number that says whether the skill earns its place.
- **pass@k** — multi-attempt reliability, reported separately from the dimension scores.
-How to read all three, and the on-disk results contract, live in [Reports & Results](reports.mdx).
+How to read all three, and the on-disk results contract, live in [Reports & Results](/skills/skillevaluator/reports).
## Explore the docs
<CardGroup cols={2}>
- <Card title="Quickstart" icon="rocket" href="quickstart.mdx">
+ <Card title="Quickstart" icon="rocket" href="/skills/skillevaluator/quickstart">
Run a bundled quality check without an LLM, or use NVIDIA Build for a rubric review.
</Card>
- <Card title="Evaluate from Your Agent" icon="robot" href="agent-skill.mdx">
+ <Card title="Evaluate from Your Agent" icon="robot" href="/skills/skillevaluator/agent-skill">
Install the SkillEvaluator skill in Codex, Claude Code, OpenCode, or OpenClaw and evaluate skills by asking.
</Card>
- <Card title="Gate Your CI" icon="shield" href="ci-integration.mdx">
+ <Card title="Gate Your CI" icon="shield" href="/skills/skillevaluator/ci-integration">
Make SkillEvaluator a merge gate — exit codes, a GitHub Actions recipe, and progressive adoption.
</Card>
- <Card title="CLI Reference" icon="terminal" href="cli-reference.mdx">
+ <Card title="CLI Reference" icon="terminal" href="/skills/skillevaluator/cli-reference">
Every command, every flag, exact defaults.
</Card>
- <Card title="Contributing" icon="book" href="developer-guide.mdx">
+ <Card title="Contributing" icon="book" href="/skills/skillevaluator/developer-guide">
Dev environment, tests, docs workflow, and how to submit a change.
</Card>
</CardGroup>
diff --git a/docs/installation.mdx b/docs/installation.mdx
index 83eaf89d..13f007b5 100644
--- a/docs/installation.mdx
+++ b/docs/installation.mdx
@@ -6,12 +6,12 @@ description: "Install SkillEvaluator with uv, from source, or in Docker, and cho
Start with the base package for deterministic quality checks, then add only the
extras needed for your next command. No API key is needed to install or to run
deterministic Tier 1 checks. LLM-backed features use a configured provider
-([Providers & Credentials](configuration.mdx)); live Tier 3 evaluation adds an
+([Providers & Credentials](/skills/skillevaluator/configuration)); live Tier 3 evaluation adds an
agent runtime and a sandbox.
For a complete first run against a bundled skill, use the
-[keyless Quickstart](quickstart.mdx#run-without-an-llm), or add the `llm` extra
-for the [NVIDIA Build route](quickstart.mdx#use-nvidia-build). Those routes use a
+[keyless Quickstart](/skills/skillevaluator/quickstart#run-without-an-llm), or add the `llm` extra
+for the [NVIDIA Build route](/skills/skillevaluator/quickstart#use-nvidia-build). Those routes use a
source checkout and `uv run`; the standalone install below is convenient when
you already have your own skill directory.
@@ -26,7 +26,7 @@ This project will download and install additional third-party open source softwa
- No API key is needed for deterministic Tier 1 checks. The base package
includes Bandit and the uv executable; Tier 1 provisions SkillSpector and
Gitleaks itself, and Semgrep is optional (see [System tools](#system-tools)). LLM-backed checks and Tiers 2–3 use a configured provider — see
- [Providers & Credentials](configuration.mdx).
+ [Providers & Credentials](/skills/skillevaluator/configuration).
## Install
@@ -50,7 +50,7 @@ Add the `llm` extra when you want to make your first rubric-evaluation call:
uv tool install --python 3.13 --reinstall "skillevaluator[llm] @ git+https://github.com/NVIDIA/SkillEvaluator.git"
```
-Then follow the [NVIDIA Build setup](quickstart.mdx#use-nvidia-build), using
+Then follow the [NVIDIA Build setup](/skills/skillevaluator/quickstart#use-nvidia-build), using
`skillevaluator rubric-eval ./my-skill` for a standalone installation. Provider
clients install with the extra; an API key is still required for the model call.
@@ -83,7 +83,7 @@ skillevaluator --version
</Tab>
<Tab title="From source" language="source">
-The [Quickstart](quickstart.mdx) uses `uv run --python 3.13` for the base quality
+The [Quickstart](/skills/skillevaluator/quickstart) uses `uv run --python 3.13` for the base quality
check and adds `--extra llm` for the NVIDIA Build rubric call. This installs
only the dependencies needed for each route.
@@ -178,7 +178,7 @@ and environment backends.
| `dev` | Build, test, coverage, and formatting tools for contributors |
<Tip>
-`[all]` installs the evaluator extras (`tier2`, `tier3`, `security`); add `dashboard` with `[all,dashboard]`. External scanner tools, Docker, agent CLIs, and the Harbor extras for managed sandbox backends (such as `harbor[modal]==0.25.0`) stay separate no matter which extra you pick — see [Agents & Sandboxes](agents-and-sandboxes.mdx#where-trials-run).
+`[all]` installs the evaluator extras (`tier2`, `tier3`, `security`); add `dashboard` with `[all,dashboard]`. External scanner tools, Docker, agent CLIs, and the Harbor extras for managed sandbox backends (such as `harbor[modal]==0.25.0`) stay separate no matter which extra you pick — see [Agents & Sandboxes](/skills/skillevaluator/agents-and-sandboxes#where-trials-run).
</Tip>
## System tools
@@ -198,7 +198,7 @@ uv tool install git+https://github.com/NVIDIA/[email protected]
brew install gitleaks # or a binary from https://github.com/gitleaks/gitleaks/releases
```
-Docker is required only for Tier 3 Docker-mode evaluation. Agent CLIs and their credentials are user-supplied — see [Agents & Sandboxes](agents-and-sandboxes.mdx).
+Docker is required only for Tier 3 Docker-mode evaluation. Agent CLIs and their credentials are user-supplied — see [Agents & Sandboxes](/skills/skillevaluator/agents-and-sandboxes).
`skillevaluator health-check --tier 1` reports the path and version of each installed scanner and exits non-zero when one is missing, misconfigured, or older than the supported release. Add `--checks security,code-integrity,dependency` to include pip-audit.
</Note>
@@ -216,7 +216,7 @@ skillevaluator --version
skillevaluator health-check
```
-`--version` confirms the CLI is on PATH — that is the whole smoke test for a Tier 1-only install. `health-check` goes one step further: a quick readiness check of the CLI and the selected live-eval backend (the agent follows the configured provider; the environment defaults to `docker`), useful once you plan to run Tier 3. On an install without the `tier3` extra or a configured provider it lists the missing pieces and exits non-zero — expected for a Tier 1-only setup, not a broken install. For a deeper diagnosis, `doctor` covers the same ground; add `doctor --verify-models` to make a live credential-bearing catalog request for each resolved agent model. A public or otherwise non-authoritative result is reported as a warning because catalog access alone does not prove inference access. See the [CLI Reference](cli-reference.mdx) for both commands.
+`--version` confirms the CLI is on PATH — that is the whole smoke test for a Tier 1-only install. `health-check` goes one step further: a quick readiness check of the CLI and the selected live-eval backend (the agent follows the configured provider; the environment defaults to `docker`), useful once you plan to run Tier 3. On an install without the `tier3` extra or a configured provider it lists the missing pieces and exits non-zero — expected for a Tier 1-only setup, not a broken install. For a deeper diagnosis, `doctor` covers the same ground; add `doctor --verify-models` to make a live credential-bearing catalog request for each resolved agent model. A public or otherwise non-authoritative result is reported as a warning because catalog access alone does not prove inference access. See the [CLI Reference](/skills/skillevaluator/cli-reference) for both commands.
## Troubleshooting
@@ -232,10 +232,10 @@ A required scanner could not run. Check that GitHub and PyPI are reachable (Skil
## Next steps
<CardGroup cols={2}>
- <Card title="Quickstart" icon="rocket" href="quickstart.mdx">
+ <Card title="Quickstart" icon="rocket" href="/skills/skillevaluator/quickstart">
Check a bundled skill without an LLM, or use NVIDIA Build for a rubric review.
</Card>
- <Card title="Providers & Credentials" icon="shield" href="configuration.mdx">
+ <Card title="Providers & Credentials" icon="shield" href="/skills/skillevaluator/configuration">
Wire up an LLM provider for Tier 2, Tier 3, and LLM-backed checks.
</Card>
</CardGroup>
diff --git a/docs/performance-dashboard.mdx b/docs/performance-dashboard.mdx
index 368ee688..2a51ea6a 100644
--- a/docs/performance-dashboard.mdx
+++ b/docs/performance-dashboard.mdx
@@ -56,7 +56,7 @@ excluded from lift, so it can differ from subtracting the displayed arm scores.
Pass-rate and resource differences can remain available when score lift is
unavailable. If every positive with-skill trial has confirmed zero activation,
the comparison is **inconclusive** and all differences stay unavailable. See
-[Skill Lift](reports.mdx#skill-lift) for these eligibility rules.
+[Skill Lift](/skills/skillevaluator/reports#skill-lift) for these eligibility rules.
When a pair's differences stay blank, the **Comparison reason** column says why.
Repeated attempts pair by case, as in the HTML report: each arm must record the
@@ -85,7 +85,7 @@ Missing measurements stay unavailable, and cost is shown only when recorded in
USD. Canonical Tier 3 JSON records each single-step trial's agent execution time
(`agent_seconds`) when Harbor's timestamps carry a timezone. Timestamps without a
timezone are treated as unavailable, as in the HTML report's
-[measured agent resources](reports.mdx#measured-agent-resources). Agent time excludes
+[measured agent resources](/skills/skillevaluator/reports#measured-agent-resources). Agent time excludes
environment setup and verification. Canonical JSON never records USD cost; open
the retained run directory to show Harbor's recorded `agent_result.cost_usd` and
multi-step totals. For retained runs whose trajectory lacks token totals or
diff --git a/docs/quickstart.mdx b/docs/quickstart.mdx
index 1bc47be0..bbaa1f00 100644
--- a/docs/quickstart.mdx
+++ b/docs/quickstart.mdx
@@ -13,7 +13,7 @@ quality check, or add NVIDIA Build to get feedback from a model.
For an executable terminal replay with checked outputs, an editable task, a
deliberate regression and report interpretation, use the
-[reproducible starter demo](starter-demo.mdx). It uses the same bundled skill.
+[reproducible starter demo](/skills/skillevaluator/starter-demo). It uses the same bundled skill.
## Get the example
@@ -54,7 +54,7 @@ This checks static quality across correctness, discoverability, reliability,
and efficiency. It does not run the security scanners or measure an agent's
performance. The report's publication status remains `INCOMPLETE` because a
quality score alone does not supply all required evaluation evidence. See
-[Reports & Results](reports.mdx) for the full output contract.
+[Reports & Results](/skills/skillevaluator/reports) for the full output contract.
## Use NVIDIA Build
@@ -96,7 +96,7 @@ reports/first-build/skillevaluator-rubric.json
Scores can vary between model calls. This evaluates the skill's instructions;
it does not run an agent with the skill. For model overrides or another
-provider, see [Providers & Credentials](configuration.mdx).
+provider, see [Providers & Credentials](/skills/skillevaluator/configuration).
## Where to go next
@@ -114,7 +114,7 @@ uv run --python 3.13 skillevaluator quality-check ./my-skill \
Full static Tier 1 coverage needs the separate
Semgrep, SkillSpector, and Gitleaks executables. Follow
-[scanner setup](installation.mdx#system-tools), then run:
+[scanner setup](/skills/skillevaluator/installation#system-tools), then run:
```bash title="Run the static Tier 1 suite"
uv run --python 3.13 skillevaluator tier1 ./my-skill --no-llm
@@ -122,18 +122,18 @@ uv run --python 3.13 skillevaluator tier1 ./my-skill --no-llm
`--no-llm` disables model-backed stages; it does not remove scanner
requirements. Missing required scanner evidence leaves validation `INCOMPLETE`
-and exits nonzero. See [Tier 1: Validation](tier1-validation.mdx) for the
+and exits nonzero. See [Tier 1: Validation](/skills/skillevaluator/tier1-validation) for the
checks and exit codes.
### Add deduplication or live evaluation
| Your goal | Additional setup | Guide |
| --- | --- | --- |
-| Find repeated or overlapping guidance | The `tier2` extra, embeddings, and chat for intra-skill analysis | [Tier 2: Deduplication](tier2-deduplication.mdx) |
-| Measure whether a skill improves an agent | The `tier3` extra, evaluator and agent credentials, a supported agent, and Docker for the default sandbox | [Tier 3: Live Evaluation](tier3-live-evaluation.mdx) |
-| Run the combined pipeline or gate CI | The extras, credentials, scanners, and runtime for the selected tiers | [CLI Reference](cli-reference.mdx#validate) and [Gate Your CI](ci-integration.mdx) |
+| Find repeated or overlapping guidance | The `tier2` extra, embeddings, and chat for intra-skill analysis | [Tier 2: Deduplication](/skills/skillevaluator/tier2-deduplication) |
+| Measure whether a skill improves an agent | The `tier3` extra, evaluator and agent credentials, a supported agent, and Docker for the default sandbox | [Tier 3: Live Evaluation](/skills/skillevaluator/tier3-live-evaluation) |
+| Run the combined pipeline or gate CI | The extras, credentials, scanners, and runtime for the selected tiers | [CLI Reference](/skills/skillevaluator/cli-reference#validate) and [Gate Your CI](/skills/skillevaluator/ci-integration) |
-The [live evaluation guide](tier3-live-evaluation.mdx#before-you-start) covers
+The [live evaluation guide](/skills/skillevaluator/tier3-live-evaluation#before-you-start) covers
runtime readiness and the first with-skill and without-skill comparison.
-[Installation](installation.mdx#choosing-extras) explains how to add each extra
+[Installation](/skills/skillevaluator/installation#choosing-extras) explains how to add each extra
to your checkout commands with `uv run --python 3.13 --extra EXTRA`.
diff --git a/docs/reports.mdx b/docs/reports.mdx
index c8e7de74..a93ef416 100644
--- a/docs/reports.mdx
+++ b/docs/reports.mdx
@@ -57,7 +57,7 @@ stops before any check and asks for an explicit `-o`/`--output-dir`.
Ownership is tied to a key stored in your home directory (see
`SKILLEVALUATOR_OUTPUT_PROVENANCE_KEY_FILE` in
-[Environment Variables](environment-variables.mdx)). Report folders written
+[Environment Variables](/skills/skillevaluator/environment-variables)). Report folders written
with another key, for example by another account or by a container whose home
directory was discarded, count as unowned. Pass `-o` in those setups.
diff --git a/docs/starter-demo.mdx b/docs/starter-demo.mdx
index 0a263410..e867425b 100644
--- a/docs/starter-demo.mdx
+++ b/docs/starter-demo.mdx
@@ -27,7 +27,7 @@ the public release wheel, not an editable development install. The 0.8.0 wheel
is available on [GitHub Releases](https://github.com/NVIDIA/SkillEvaluator/releases/tag/v0.8.0);
do not assume the same version is already on PyPI. Installation needs
internet access. The demo then runs locally. For standalone pip installations
-and supported Python versions, see [Installation](installation.mdx).
+and supported Python versions, see [Installation](/skills/skillevaluator/installation).
The runner prints each command and its real terminal output. It copies the
installed skill into `reports/starter/text-analyzer`, stages the public input
@@ -69,7 +69,7 @@ precedence over missing evidence. Do not interpret incomplete evidence as a
zero score, a pass, or proof of a skill defect. Fix execution/setup problems and
rerun before drawing performance conclusions. For live agent results, compare
with-skill and without-skill outcomes, inspect failed attempts and coverage, and
-use the verdict and completion fields together; see [Reports & Results](reports.mdx).
+use the verdict and completion fields together; see [Reports & Results](/skills/skillevaluator/reports).
### Replay a single command manually
@@ -87,8 +87,8 @@ prebuilt report. The usual quality threshold is 70; this demo deliberately uses
## Add model feedback or a live agent
For fresh model-backed instruction feedback on this same skill, configure an
-account and key using [NVIDIA Build](quickstart.mdx#use-nvidia-build) or another
-[public provider](configuration.mdx). Never put a key in a report or fixture.
+account and key using [NVIDIA Build](/skills/skillevaluator/quickstart#use-nvidia-build) or another
+[public provider](/skills/skillevaluator/configuration). Never put a key in a report or fixture.
Then run:
```bash
@@ -99,7 +99,7 @@ uv run --isolated --no-project --python 3.13 --with "skillevaluator[llm] @ https
Model catalogs and account access change. A visible catalog entry does not
guarantee inference access. If the default model returns HTTP 404 or 410, select
a model available to your account with `SKILL_EVAL_LLM_MODEL` as described in
-[configuration](configuration.mdx). Authentication errors, rate limits, and
+[configuration](/skills/skillevaluator/configuration). Authentication errors, rate limits, and
unavailable models are not low rubric scores. Inspect the JSON findings for
`llm_unavailable`; do not present a fallback response as a completed evaluation.
@@ -114,7 +114,7 @@ supported model rather than interpreting it as a completed low-scoring run.
Rubric feedback is not agent uplift. For the paired agent experiment, keep the
same copied task and input and follow the
-[live evaluation prerequisites](tier3-live-evaluation.mdx#before-you-start).
+[live evaluation prerequisites](/skills/skillevaluator/tier3-live-evaluation#before-you-start).
After installing the `tier3` extra, configuring evaluator/agent credentials and
starting Docker, run:
@@ -127,7 +127,7 @@ The authored task is `evals/evals.json`; `evals/files/input.txt` is staged into
the sandbox at `/workspace/input/input.txt`. Read both before paying for agent
runs. One tiny task is a plumbing example, not representative coverage or
independent validation of a grader. Model and sandbox usage can incur charges.
-Read [Eval Datasets](eval-datasets.mdx) for additional cases, grading and policy,
+Read [Eval Datasets](/skills/skillevaluator/eval-datasets) for additional cases, grading and policy,
and retain all completed attempts and errors rather than publishing only a score.
## Adapt the example
@@ -162,15 +162,15 @@ self-checks are not a substitute for feedback from a first-time human user.
| Symptom | Action |
| --- | --- |
-| Python or dependencies cannot download | Check internet/proxy access and uv installation; see [installation](installation.mdx). No key is needed for the keyless route. |
+| Python or dependencies cannot download | Check internet/proxy access and uv installation; see [installation](/skills/skillevaluator/installation). No key is needed for the keyless route. |
| Output directory already exists | Keep it as evidence; use a new `--output` path. Nothing is overwritten. |
| A checked score or report field differs | Read the retained JSON/text, confirm `receipt.json` if present and the installed version; do not blindly update expectations. A failed demo does not write a success receipt. |
| Analysis count differs | Check file contents, encoding and newlines; the input is UTF-8 plain text with a final newline. |
-| Scanner or sandbox is missing | The keyless demo does not need either. Follow [scanner setup](installation.mdx#system-tools) or [agent setup](agents-and-sandboxes.mdx) before adding those stages. |
+| Scanner or sandbox is missing | The keyless demo does not need either. Follow [scanner setup](/skills/skillevaluator/installation#system-tools) or [agent setup](/skills/skillevaluator/agents-and-sandboxes) before adding those stages. |
| Template reference asks you to sign in | Use the [public template reference](https://github.com/NVIDIA/SkillEvaluator/blob/main/ci/templates/README.md) and [versioned bundles](https://github.com/NVIDIA/SkillEvaluator/tree/main/ci/bundles/0.8.0), not a private mirror. Check bundle files against `SHA256SUMS.txt`. |
The public template reference and versioned downloadable bundles were consolidated
in [the public template update](https://github.com/NVIDIA/SkillEvaluator/pull/328).
They require no private account. For repository automation, continue with
-[Repository Integration](repository-integration.mdx); for improvements to this
-starter, see [Contributing](developer-guide.mdx).
+[Repository Integration](/skills/skillevaluator/repository-integration); for improvements to this
+starter, see [Contributing](/skills/skillevaluator/developer-guide).
diff --git a/docs/tier1-validation.mdx b/docs/tier1-validation.mdx
index 5508da2c..68f0fb20 100644
--- a/docs/tier1-validation.mdx
+++ b/docs/tier1-validation.mdx
@@ -16,9 +16,9 @@ runs rubric judging, LLM security analysis, and finding verification. Without a
provider, those LLM stages are explicitly reported as skipped. Use `--no-llm`
for static checks only, or `--llm` to require a provider instead of allowing a
keyless run. Full scanner coverage still requires the external tools listed in
-[Installation](installation.mdx#system-tools).
+[Installation](/skills/skillevaluator/installation#system-tools).
-The two-variable [provider setup](configuration.mdx) is enough to configure the
+The two-variable [provider setup](/skills/skillevaluator/configuration) is enough to configure the
LLM stages for NVIDIA Build or OpenAI. Calling `skillevaluator tier1` without a
path shows help; expert subcommands remain available below.
@@ -36,8 +36,8 @@ when it changes a skill. Eligible pushes and manual runs on the default branch
run them for the whole collection.
Existing released Tier 1-only consumers keep their current workflow and
-bundled image pins. See [GitHub Actions](github-actions.mdx) or
-[GitLab CI/CD](gitlab-ci.mdx) to choose either path.
+bundled image pins. See [GitHub Actions](/skills/skillevaluator/github-actions) or
+[GitLab CI/CD](/skills/skillevaluator/gitlab-ci) to choose either path.
## What Tier 1 checks
@@ -68,7 +68,7 @@ for skills. Its Tier 1 LLM checks still run only when requested.
skillevaluator validate ./my-skill
```
- By default you get a compact pipeline view in the terminal — a per-check ticker per tier — and the run writes HTML and JSON report files unless you pass `-r` explicitly, which is then honored exactly (including `-r cli` for terminal-only output). Add `--verbose` for the full per-check detail stream instead of the compact view. The full pipeline needs the [provider and runtime setup](quickstart.mdx); use `--tiers 1` for static checks only. The exit code is `0` when every gate passes and `1` when one fails.
+ By default you get a compact pipeline view in the terminal — a per-check ticker per tier — and the run writes HTML and JSON report files unless you pass `-r` explicitly, which is then honored exactly (including `-r cli` for terminal-only output). Add `--verbose` for the full per-check detail stream instead of the compact view. The full pipeline needs the [provider and runtime setup](/skills/skillevaluator/quickstart); use `--tiers 1` for static checks only. The exit code is `0` when every gate passes and `1` when one fails.
### Run the complete offline gate
@@ -108,7 +108,7 @@ for skills. Its Tier 1 LLM checks still run only when requested.
Direct `tier1 PATH` and `tier1 validate PATH` print the terminal report and
write JSON and HTML reports by default. Use `-r cli` for terminal-only output.
-Reports for any of these runs are selected with `-r`/`--report` (`cli`, `json`, `html`, `markdown`, `sarif`) and written to `-o`/`--output-dir` (default `reports/`). File naming, `BENCHMARK.md`, and the JSON contract are covered in [Reports & Results](reports.mdx).
+Reports for any of these runs are selected with `-r`/`--report` (`cli`, `json`, `html`, `markdown`, `sarif`) and written to `-o`/`--output-dir` (default `reports/`). File naming, `BENCHMARK.md`, and the JSON contract are covered in [Reports & Results](/skills/skillevaluator/reports).
## Content types
@@ -744,7 +744,7 @@ The command line rejects a malformed waiver file before the run starts, with exi
## Scanner setup
-Bandit installs with SkillEvaluator; install `skillevaluator[security]` for pip-audit, which the opt-in dependency audit uses. Install Semgrep and SkillSpector in separate tool environments, and install Gitleaks separately — it is a Go binary, not a pip package. See [Installation](installation.mdx) for commands and the extras table.
+Bandit installs with SkillEvaluator; install `skillevaluator[security]` for pip-audit, which the opt-in dependency audit uses. Install Semgrep and SkillSpector in separate tool environments, and install Gitleaks separately — it is a Go binary, not a pip package. See [Installation](/skills/skillevaluator/installation) for commands and the extras table.
Semgrep uses a policy packaged with the installed distribution, with metrics and version checks disabled; Tier 1 never fetches a registry policy at runtime. The other public Tier 1 configuration also ships inside the package under `skillevaluator/config/`: `pii_patterns.yaml`, `unicode_smuggle_patterns.yaml`, `license_config.yaml`, and `profiles/default.yaml`.
@@ -815,7 +815,7 @@ Markdown and HTML reports show the same outcome as a `skillspector_baseline` lin
Tier 1 checks always gate the exit code inside `validate`. Tier 2 gates when it runs unless `--no-block-on-dedup` is set. Tier 3 is advisory unless `--block-on-agent-eval` is set. The `quality` check requires the composite score to reach `--min-score` (default `70`) as well as no HIGH or CRITICAL quality findings under the active policy. Severity overrides cannot waive the minimum. Reports record the effective gate membership for every tier so saved evidence and process status agree.
-`--fail-fast` stops after the first failed check. `-c`/`--continue-on-failure` overrides it, records the full pipeline, and keeps scanning a collection past a CRITICAL finding. Wiring these exit codes into a merge gate — including a complete GitHub Actions recipe — is covered in [Gate Your CI](ci-integration.mdx).
+`--fail-fast` stops after the first failed check. `-c`/`--continue-on-failure` overrides it, records the full pipeline, and keeps scanning a collection past a CRITICAL finding. Wiring these exit codes into a merge gate — including a complete GitHub Actions recipe — is covered in [Gate Your CI](/skills/skillevaluator/ci-integration).
## Troubleshooting
@@ -906,7 +906,7 @@ Tier 1 checks always gate the exit code inside `validate`. Tier 2 gates when it
Each stage gets the full watchdog, so a scan that stalls in both stages can
take up to twice the value before it is reported. See
- [Environment Variables](environment-variables.mdx#validation-tier-1).
+ [Environment Variables](/skills/skillevaluator/environment-variables#validation-tier-1).
</Accordion>
<Accordion title="The SkillSpector LLM provider keeps failing">
With `--llm`, SkillSpector v2.12.0 and later retries a failing provider call
@@ -979,16 +979,16 @@ Tier 1 checks always gate the exit code inside `validate`. Tier 2 gates when it
## Next steps
<CardGroup cols={2}>
- <Card title="Tier 2: Deduplication" icon="copy" href="tier2-deduplication.mdx">
+ <Card title="Tier 2: Deduplication" icon="copy" href="/skills/skillevaluator/tier2-deduplication">
Check whether your skill repeats itself or overlaps the rest of your collection.
</Card>
- <Card title="Gate Your CI" icon="shield" href="ci-integration.mdx">
+ <Card title="Gate Your CI" icon="shield" href="/skills/skillevaluator/ci-integration">
Turn the Tier 1 exit code into a merge gate with a copy-paste workflow.
</Card>
- <Card title="CLI Reference" icon="terminal" href="cli-reference.mdx">
+ <Card title="CLI Reference" icon="terminal" href="/skills/skillevaluator/cli-reference">
Every flag and default for validate and the standalone Tier 1 commands.
</Card>
- <Card title="Environment Variables" icon="book" href="environment-variables.mdx">
+ <Card title="Environment Variables" icon="book" href="/skills/skillevaluator/environment-variables">
Profile, scanner override, and identity variables in one table.
</Card>
</CardGroup>
diff --git a/docs/tier2-deduplication.mdx b/docs/tier2-deduplication.mdx
index e26fd3e1..760ce098 100644
--- a/docs/tier2-deduplication.mdx
+++ b/docs/tier2-deduplication.mdx
@@ -8,8 +8,8 @@ Tier 2 finds redundant content — inside a single skill and across a skill
collection — using embedding similarity, with chat-LLM verification for
intra-skill candidates. It needs a configured embeddings provider; intra-skill
analysis also needs a chat LLM. See
-[Providers & Credentials](configuration.mdx) to set both up, including the
-[fully local recipe](configuration.mdx#fully-local-setup) when skill content
+[Providers & Credentials](/skills/skillevaluator/configuration) to set both up, including the
+[fully local recipe](/skills/skillevaluator/configuration#fully-local-setup) when skill content
must not leave your machine.
Both checks read local files only.
@@ -21,7 +21,7 @@ The [all-tier onboarding path](/skills/skillevaluator/advanced-ci)
runs Tier 2 as an advisory check, independently from the Tier 1 image and paired
Tier 3 evaluation. Configure chat and embedding provider credentials as CI
secrets. Tier 2 does not read `evals/`; committed native datasets are needed
-only for [Tier 3](tier3-live-evaluation.mdx). Missing credentials or a central
+only for [Tier 3](/skills/skillevaluator/tier3-live-evaluation). Missing credentials or a central
dataset mapping produce an actionable incomplete result before paid work.
The aggregate artifact preserves native Tier 2 JSON and HTML, including
@@ -55,7 +55,7 @@ Direct Tier 2 prints the terminal report and writes JSON and HTML reports to
For NVIDIA Build or OpenAI, the same provider and key configure both chat and
embeddings; no separate embedding variables are needed. See
-[Providers & Credentials](configuration.mdx) for other providers.
+[Providers & Credentials](/skills/skillevaluator/configuration) for other providers.
## Which check do you need
@@ -242,7 +242,7 @@ again on a rerun, so Tier 2 records a MEDIUM `llm_refused` finding that names
the overlapping sections and does not block. Review that overlap by hand, or
rerun with another chat model (`--llm-model` or `SKILL_EVAL_LLM_MODEL`). To make
a refusal block `validate`, raise `CONTENT_DEDUP.llm_refused` to `high` in a
-[policy overlay](tier1-validation.mdx#validation-profiles-and-policies).
+[policy overlay](/skills/skillevaluator/tier1-validation#validation-profiles-and-policies).
The JSON report's `llm_analysis` names the configured `provider` and `model`.
A server-side fallback model can answer a cluster in the configured model's
@@ -482,7 +482,7 @@ All Tier 2 commands accept the standard report options (`-r cli,json,html,markdo
and `-o`). HIGH and CRITICAL findings, along with blocking analysis errors,
produce a non-zero exit code; MEDIUM and LOW findings are advisory. Exit
behavior is identical in every output mode, so these commands work as CI gates
-— see [Gate Your CI](ci-integration.mdx).
+— see [Gate Your CI](/skills/skillevaluator/ci-integration).
Inside the combined `validate` pipeline, Tier 2 also gates by default. Pass
`--no-block-on-dedup` to keep the scan and its findings in every report without
@@ -559,7 +559,7 @@ scan.
- Catalog files are read and written locally; SkillEvaluator does not upload
them to a catalog service.
-Use the [fully local provider recipe](configuration.mdx#fully-local-setup) when
+Use the [fully local provider recipe](/skills/skillevaluator/configuration#fully-local-setup) when
skill content must not leave the machine, and review your provider's data
handling terms before sending confidential skill content to a hosted endpoint.
@@ -604,13 +604,13 @@ handling terms before sending confidential skill content to a hosted endpoint.
## Next steps
<CardGroup cols={2}>
- <Card title="Tier 3: Live Evaluation" icon="flask" href="tier3-live-evaluation.mdx">
+ <Card title="Tier 3: Live Evaluation" icon="flask" href="/skills/skillevaluator/tier3-live-evaluation">
Prove the skill changes agent behavior with a live A/B evaluation.
</Card>
- <Card title="Providers & Credentials" icon="book" href="configuration.mdx">
+ <Card title="Providers & Credentials" icon="book" href="/skills/skillevaluator/configuration">
Configure an embeddings provider — including a fully local setup.
</Card>
- <Card title="CLI Reference" icon="terminal" href="cli-reference.mdx">
+ <Card title="CLI Reference" icon="terminal" href="/skills/skillevaluator/cli-reference">
Every Tier 2 flag, default, and report option.
</Card>
</CardGroup>
diff --git a/docs/tier3-live-evaluation.mdx b/docs/tier3-live-evaluation.mdx
index cd484692..f03941bc 100644
--- a/docs/tier3-live-evaluation.mdx
+++ b/docs/tier3-live-evaluation.mdx
@@ -2184,19 +2184,19 @@ machine-readable JSON contract, and how to judge whether a given lift is good.
## Next steps
<CardGroup cols={2}>
- <Card title="Eval Datasets" icon="book" href="eval-datasets.mdx">
+ <Card title="Eval Datasets" icon="book" href="/skills/skillevaluator/eval-datasets">
The full evals/ contract — dataset format, generation, and config.yml run
policy.
</Card>
- <Card title="Agents & Sandboxes" icon="shield" href="agents-and-sandboxes.mdx">
+ <Card title="Agents & Sandboxes" icon="shield" href="/skills/skillevaluator/agents-and-sandboxes">
Every supported agent and execution environment, plus readiness and
debugging tools.
</Card>
- <Card title="Custom Graders & Tasks" icon="flask" href="custom-graders.mdx">
+ <Card title="Custom Graders & Tasks" icon="flask" href="/skills/skillevaluator/custom-graders">
Bring your own grader or a complete Harbor task when the defaults can't
express success.
</Card>
- <Card title="Reports & Results" icon="terminal" href="reports.mdx">
+ <Card title="Reports & Results" icon="terminal" href="/skills/skillevaluator/reports">
What every run writes to disk and how to read it — including the JSON
contract for CI.
</Card>
diff --git a/docs/trace-datasets.mdx b/docs/trace-datasets.mdx
index 0b94b4c6..796626fd 100644
--- a/docs/trace-datasets.mdx
+++ b/docs/trace-datasets.mdx
@@ -118,4 +118,4 @@ is detected. Reviewers must still inspect all bundle files for sensitive data,
missing replay inputs, and suitability for publication. Review decisions are
explicit local metadata, not authenticated signatures or proof of who reviewed.
-See [Eval Datasets](eval-datasets.mdx) for the promoted dataset contract.
+See [Eval Datasets](/skills/skillevaluator/eval-datasets) for the promoted dataset contract.