Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
Python agent-evaluationagent-pluginsagent-securityagent-skillsagentic-aibenchmarkci-templatesclaude-codecodexevaluateevaluationharnesspluginplugin-evaluationsecurity-scannerskill-evalskill-evaluationskill-evaluatorskill-gap-analysisskills
docs: repair onboarding page navigation (#573)
Christopher Kevina0affcc9fe43
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex f8ac0786..3917cd97 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -256,6 +256,9 @@ All notable changes to SkillEvaluator are documented in this file.successful stop, instead of reporting an interrupted cleanup on cancellation.- Native NeMo Gym cleanup now also stops run-owned detached server processesafter startup timeouts, interruptions and early exits, without targeting other runs.+- Published documentation links use page routes so readers can navigate between+ setup, report interpretation, recovery and the reference pages.+- NeMo Gym results reject verifier rewards that identify a different stagedcase and retain custom grader explanations in collected results.- The Copilot CLI `CP-NO-FRONTMATTER` finding and the plugin component findingsdiff --git a/docs/agent-skill.mdx b/docs/agent-skill.mdxindex 18bc447e..b11676a6 100644--- a/docs/agent-skill.mdx+++ b/docs/agent-skill.mdx@@ -48,7 +48,7 @@ pip install --upgrade skillevaluator # Tier 1 and 2pip install --upgrade "skillevaluator[all]" # adds Tier 3 live evaluation```-See [Installation](installation.mdx) for extras and Python versions.+See [Installation](/skills/skillevaluator/installation) for extras and Python versions.### Add the skill to your agent@@ -100,7 +100,7 @@ export NVIDIA_API_KEY=...```OpenAI, Anthropic, and any OpenAI-compatible gateway also work; see-[Providers & Credentials](configuration.mdx). Keys stay in your environment; the+[Providers & Credentials](/skills/skillevaluator/configuration). Keys stay in your environment; theskill tells the agent never to print or store them.</Steps>@@ -161,7 +161,7 @@ since been removed.<Accordion title="Tier 3: in a sandbox">Docker is the default backend; SkillEvaluator drives Harbor for you. For trusted skills,`--env-mode local` runs the agent on the host under Bubblewrap (Linux) or Seatbelt (macOS).- See [Agents & Sandboxes](agents-and-sandboxes.mdx).+ See [Agents & Sandboxes](/skills/skillevaluator/agents-and-sandboxes).</Accordion><Accordion title="Incomplete runs">If an attempt errors or times out, the run is INCOMPLETE and no final score or lift is@@ -173,16 +173,16 @@ since been removed.## Next steps<CardGroup cols={2}>- <Card title="Tier 3: Live Evaluation" icon="flask" href="tier3-live-evaluation.mdx">+ <Card title="Tier 3: Live Evaluation" icon="flask" href="/skills/skillevaluator/tier3-live-evaluation">How baseline and with-skill runs are scored, and what Skill Lift means.</Card>- <Card title="Eval Datasets" icon="list-check" href="eval-datasets.mdx">+ <Card title="Eval Datasets" icon="list-check" href="/skills/skillevaluator/eval-datasets">The `evals.json` contract the agent writes and validates.</Card>- <Card title="Reports & Results" icon="chart-column" href="reports.mdx">+ <Card title="Reports & Results" icon="chart-column" href="/skills/skillevaluator/reports">Every report file the agent reads, and what each field means.</Card>- <Card title="Bring It to Your Repository" icon="code-branch" href="repository-integration.mdx">+ <Card title="Bring It to Your Repository" icon="code-branch" href="/skills/skillevaluator/repository-integration">Check every skill change in GitHub or GitLab CI.</Card></CardGroup>diff --git a/docs/agents-and-sandboxes.mdx b/docs/agents-and-sandboxes.mdxindex 361deda7..6426f5eb 100644--- a/docs/agents-and-sandboxes.mdx+++ b/docs/agents-and-sandboxes.mdx@@ -8,7 +8,7 @@ run: which agents attempt the tasks (`--agents`) and where the trials execute(`--env-mode`). This page is the complete matrix for both, plus the resource,safety, and workspace controls around them and the readiness checks that tellyou a run will work before you spend compute. The run workflow itself is-[Tier 3: Live Evaluation](tier3-live-evaluation.mdx).+[Tier 3: Live Evaluation](/skills/skillevaluator/tier3-live-evaluation).[Harbor](https://github.com/harbor-framework/harbor) is the open-source agentevaluation framework that executes each Tier 3 trial and provides its Docker,@@ -147,7 +147,7 @@ refusal in 20 cases with 3 attempts each, the run's `result.json` recordsdoes `validate --block-on-agent-eval`. On a route that serves the fallbackmodel, such as the Anthropic API, the retry used to finish such an attempt,and the attempt was scored, so a run that completed before can now fail. See-[Troubleshooting](tier3-live-evaluation.mdx#troubleshooting) for what to do.+[Troubleshooting](/skills/skillevaluator/tier3-live-evaluation#troubleshooting) for what to do.The `claude` alias is canonicalized to `claude-code` everywhere an agentname can appear — `-a/--agents`, `--agent-model`, and `harbor.agents` in@@ -166,7 +166,7 @@ skillevaluator tier3 evaluate ./my-skill --agents codex --env-mode docker \```To make an override persistent, set it in `evals/config.yml` under-`harbor.agents.<name>.model` — see [Eval Datasets](eval-datasets.mdx).+`harbor.agents.<name>.model` — see [Eval Datasets](/skills/skillevaluator/eval-datasets).<Note>Harbor installs agent CLIs in Docker and remote task environments. Local mode@@ -176,7 +176,7 @@ Operator credentials come only from the host environment — acredential name (`ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`,`CLAUDE_CODE_USE_BEDROCK`, `NVIDIA_API_KEY`, `OPENAI_API_KEY`,`OPENAI_BASE_URL`) or references one with `${...}` is rejected at run time.-See the two credential roles in [Providers & Credentials](configuration.mdx).+See the two credential roles in [Providers & Credentials](/skills/skillevaluator/configuration).</Note>## One NVIDIA Build key for everything@@ -601,7 +601,7 @@ skillevaluator tier3 evaluate ./my-skill --agents codex --env-mode docker \To make the sizing part of the skill's run policy instead of a flag, set`harbor.resources` (`cpus`, `memory_mb`, `storage_mb`) in `evals/config.yml`-— see [Eval Datasets](eval-datasets.mdx). A trial that dies mid-run with the+— see [Eval Datasets](/skills/skillevaluator/eval-datasets). A trial that dies mid-run with theagent process killed is the classic out-of-memory symptom; raise`--override-memory-mb` first.@@ -637,7 +637,7 @@ skips the check.Environment rules — size limits, auto-generated fallbacks, and sidecarcontainers — are part of the `evals/` contract in-[Eval Datasets](eval-datasets.mdx).+[Eval Datasets](/skills/skillevaluator/eval-datasets).Docker Compose project `.env` files are rejected before startup. Protectedmain-container and sidecar operations disable Compose's implicit env-file@@ -746,7 +746,7 @@ The sandbox policy is controlled by environment variables:| `SKILLEVALUATOR_RUNTIME_DIR` | `~/.local/share/skillevaluator/runtimes` | Where local mode looks for managed agent CLIs. Must be a dedicated subdirectory — not the home directory or one of its parents. |Full semantics for each variable are in-[Environment Variables](environment-variables.mdx). `doctor --env-mode local`+[Environment Variables](/skills/skillevaluator/environment-variables). `doctor --env-mode local`verifies the sandbox backend and the installed agent CLIs before you commit toa run.@@ -867,7 +867,7 @@ any check fails, so either can gate a script or CI job. Warnings frominconclusive rather than a definitive rejection. Both commands check Tier 3 bydefault. `--tier 1` checks the Tier 1 scanners, `--tier 2` the embedding and LLMproviders, and `--tier all` every tier; see the-[CLI Reference](cli-reference.mdx#doctor).+[CLI Reference](/skills/skillevaluator/cli-reference#doctor).### doctor@@ -925,16 +925,16 @@ skillevaluator harbor-view <jobs-dir>The retained tree holds the full agent trajectory and grader output for everytrial — the fastest way to see what an agent actually did. The results layoutand where `<jobs-dir>` lands on disk are in-[Reports & Results](reports.mdx); debugging custom graders this way is-covered in [Custom Graders & Tasks](custom-graders.mdx).+[Reports & Results](/skills/skillevaluator/reports); debugging custom graders this way is+covered in [Custom Graders & Tasks](/skills/skillevaluator/custom-graders).## Next steps<CardGroup cols={2}>- <Card title="Tier 3: Live Evaluation" icon="flask" href="tier3-live-evaluation.mdx">+ <Card title="Tier 3: Live Evaluation" icon="flask" href="/skills/skillevaluator/tier3-live-evaluation">The end-to-end run workflow — readiness, dataset, evaluation, and results.</Card>- <Card title="Custom Graders & Tasks" icon="terminal" href="custom-graders.mdx">+ <Card title="Custom Graders & Tasks" icon="terminal" href="/skills/skillevaluator/custom-graders">Bring your own grader or Harbor task when the defaults can't expresssuccess.</Card>diff --git a/docs/cli-reference.mdx b/docs/cli-reference.mdxindex a1cb1fee..62a2ca12 100644--- a/docs/cli-reference.mdx+++ b/docs/cli-reference.mdx@@ -154,7 +154,7 @@ errors, such as an option the workflow does not accept, exit `2`.| `--quality-config PATH` | packaged `quality-default.yaml` | YAML file that adjusts the quality check's expected headings and error-handling rules; settings it omits keep their packaged values. See [Tier 1: Validation](/skills/skillevaluator/tier1-validation#the-checks). || `--pii-waivers PATH` | none | Reviewed, expiring PII waivers applied by the `pii` check. See [PII waivers](/skills/skillevaluator/tier1-validation#pii-waivers). || `--profile TEXT` | `SKILLEVALUATOR_PROFILE`, then `default` | Validation policy: `default` or `registry`. `external` is a compatibility alias for `default`. |-| `--policy FILE` | none | Explicit policy overlay, including reviewed ownership-contact severity. See [Tier 1 policies](tier1-validation.mdx#validation-profiles-and-policies). |+| `--policy FILE` | none | Explicit policy overlay, including reviewed ownership-contact severity. See [Tier 1 policies](/skills/skillevaluator/tier1-validation#validation-profiles-and-policies). || `--previous-version TEXT` | `SKILLEVALUATOR_PREVIOUS_VERSION`, else unset | Previous version for version checks || `--skillspector-baseline FILE` | unset | Audited SkillSpector suppression baseline; never discovered from the skill. See [Audited SkillSpector baselines](/skills/skillevaluator/tier1-validation#audited-skillspector-baselines). |diff --git a/docs/configuration.mdx b/docs/configuration.mdxindex 2b65cad4..63ee8d4b 100644--- a/docs/configuration.mdx+++ b/docs/configuration.mdx@@ -345,10 +345,10 @@ Setting `SKILLSPECTOR_PROVIDER` yourself overrides the bridge, but it must name## Next steps<CardGroup cols={2}>- <Card title="Tier 2: Deduplication" icon="copy" href="tier2-deduplication.mdx">+ <Card title="Tier 2: Deduplication" icon="copy" href="/skills/skillevaluator/tier2-deduplication">Put your embedding provider to work — find overlap within and across skills.</Card>- <Card title="Tier 3: Live Evaluation" icon="flask" href="tier3-live-evaluation.mdx">+ <Card title="Tier 3: Live Evaluation" icon="flask" href="/skills/skillevaluator/tier3-live-evaluation">Run a live agent evaluation with the two-credential setup above.</Card></CardGroup>diff --git a/docs/custom-graders.mdx b/docs/custom-graders.mdxindex 28f78008..161f0a41 100644--- a/docs/custom-graders.mdx+++ b/docs/custom-graders.mdx@@ -10,7 +10,7 @@ your own complete [Harbor](https://github.com/harbor-framework/harbor) task.Harbor is the open-source agent evaluation framework that executes Tier 3trials. Scaffolding is keyless and takes one command; running a custom-gradedevaluation needs the same Tier 3 stack as any live eval — see-[Tier 3: Live Evaluation](tier3-live-evaluation.mdx).+[Tier 3: Live Evaluation](/skills/skillevaluator/tier3-live-evaluation).## Grading modes@@ -88,7 +88,7 @@ optional — the config already selects it.`custom_only` still requires a numeric overall score. Qualitative reasons canaccompany it, but prose alone is not a reward. For a manual, evidence-linkedassessment alongside current scores, see the-[qualitative assessment review contract](reports.mdx#qualitative-assessment-review-contract).+[qualitative assessment review contract](/skills/skillevaluator/reports#qualitative-assessment-review-contract).Its worksheet is a documentation prototype, not input to your grader or ascoreless runtime mode.@@ -172,7 +172,7 @@ Role log directories retain their host locations, so usederiving a parent from a role directory or writing `../artifacts` from it.Your grader reads Harbor's `trajectory.json` as written. Unlike the standard-grader, it does not [rebuild a missing or empty trajectory](tier3-live-evaluation.mdx#the-graded-trajectory)+grader, it does not [rebuild a missing or empty trajectory](/skills/skillevaluator/tier3-live-evaluation#the-graded-trajectory)from agent logs, so handle that case explicitly. A step's `message` and anobservation result's `content` can each be a string or a list of content parts(`text`, `image`, or `audio`), so read both forms.@@ -196,7 +196,7 @@ skillevaluator init-harbor-task ./my-skill| `--force` | off | Overwrite an existing starter case |Exact defaults for both scaffolding commands are in the-[CLI Reference](cli-reference.mdx#init-custom-grader).+[CLI Reference](/skills/skillevaluator/cli-reference#init-custom-grader).The scaffold produces a complete, runnable task source:@@ -235,7 +235,7 @@ Two rules keep BYOT predictable:`[metadata]` in `task.toml`, falling back to the case directory name — mustmatch an entry `id` in `evals/evals.json`, or the run refuses to start.`custom_only` has no such requirement. See- [Eval Datasets](eval-datasets.mdx).+ [Eval Datasets](/skills/skillevaluator/eval-datasets).- **Your source is never mutated in place.** SkillEvaluator stages with-skilland baseline copies of `evals/harbor/` under the results directory for eachrun; the files you author stay untouched. In a paired run, the staged copies@@ -310,7 +310,7 @@ skillevaluator harbor-view <jobs-dir>Grader stdout, the written `reward.json`, and the full trajectory are all inthe retained job tree — the fastest way to find out why a score isn't what-you expected. [Agents & Sandboxes](agents-and-sandboxes.mdx) covers the+you expected. [Agents & Sandboxes](/skills/skillevaluator/agents-and-sandboxes) covers thedebugging workflow in depth.## What reports show@@ -334,7 +334,7 @@ Custom scores flow into the same results tree as everything else:run's attempt policy.Custom lift needs both arms — a `--skip-baseline` run reports your custom-scores but no lift. [Reports & Results](reports.mdx) covers the full on-disk+scores but no lift. [Reports & Results](/skills/skillevaluator/reports) covers the full on-disklayout and the machine-readable contract.## Import precomputed custom metrics@@ -409,7 +409,7 @@ must have a nonempty `custom_metrics` object, but its declared metrics may allbe null. Arm means exclude missing values and report a scored count per metricagainst total observations. All-missing metrics render `NO SCORE`.-Case IDs are strings following the [dataset identity rules](eval-datasets.mdx). Attempts are+Case IDs are strings following the [dataset identity rules](/skills/skillevaluator/eval-datasets). Attempts areintegers from 1 through 1000, not booleans. Only `with_skill` and `without_skill`arms are accepted; duplicate case/attempt/arm identities and case IDs differingonly in capitalization are rejected. Standard metric names, metadata names@@ -480,11 +480,11 @@ scores produce null delta/improvement and `NO SCORE`, not a fabricated zero.## Next steps<CardGroup cols={2}>- <Card title="Eval Datasets" icon="book" href="eval-datasets.mdx">+ <Card title="Eval Datasets" icon="book" href="/skills/skillevaluator/eval-datasets">The full evals/ contract — dataset format, config.yml run policy, andfixtures your custom task can build on.</Card>- <Card title="Reports & Results" icon="terminal" href="reports.mdx">+ <Card title="Reports & Results" icon="terminal" href="/skills/skillevaluator/reports">Everything a run writes to disk and how to read it, including wherecustom metrics and custom lift surface.</Card>diff --git a/docs/developer-guide.mdx b/docs/developer-guide.mdxindex 6cef57e0..b29ef1b7 100644--- a/docs/developer-guide.mdx+++ b/docs/developer-guide.mdx@@ -93,7 +93,7 @@ The script renders its own fixture reports, then writes screenshots and `navigat## Documentation and starter changes-The [terminal starter](starter-demo.mdx) uses a public skill, task and input with+The [terminal starter](/skills/skillevaluator/starter-demo) uses a public skill, task and input withreal CLI output checks. For changes to it, run:```bash@@ -166,7 +166,7 @@ Contributions flow through GitHub pull requests:2. **Keep each pull request focused.** Use the pull request template and explain the user-visible behavior and the verification you performed.3. **Add focused tests** for behavioral changes.4. **Update [CHANGELOG.md](https://github.com/NVIDIA/SkillEvaluator/blob/main/CHANGELOG.md)** when the change affects users.-5. **Keep provider credentials, customer data, and private benchmark material out of the repository.** Tier 3 live evaluation reads provider and agent credentials from your environment, never from the repo (with NVIDIA Build, one `NVIDIA_API_KEY` covers both roles) — see [Providers & Credentials](configuration.mdx).+5. **Keep provider credentials, customer data, and private benchmark material out of the repository.** Tier 3 live evaluation reads provider and agent credentials from your environment, never from the repo (with NVIDIA Build, one `NVIDIA_API_KEY` covers both roles) — see [Providers & Credentials](/skills/skillevaluator/configuration).<Warning>Every commit needs a Developer Certificate of Origin sign-off — commit with `git commit -s`. Pull requests containing unsigned commits are not accepted. By contributing, you confirm you have the right to submit the work under the Apache License 2.0.diff --git a/docs/environment-variables.mdx b/docs/environment-variables.mdxindex 3c3e1db1..9c79373d 100644--- a/docs/environment-variables.mdx+++ b/docs/environment-variables.mdx@@ -21,7 +21,7 @@ contract. None belongs to either SkillEvaluator prefix.## LLM provider-These variables select and configure the provider used for LLM-backed checks and judging. The setup walkthrough, including per-provider export blocks and default models, is in [Providers & Credentials](configuration.mdx).+These variables select and configure the provider used for LLM-backed checks and judging. The setup walkthrough, including per-provider export blocks and default models, is in [Providers & Credentials](/skills/skillevaluator/configuration).| Variable | Default | Effect || --- | --- | --- |@@ -116,7 +116,7 @@ The `--llm` security analysis runs [SkillSpector](https://github.com/NVIDIA/Skil## Embeddings-Tier 2 semantic checks use an OpenAI-compatible embeddings API. Workflow context lives in [Providers & Credentials](configuration.mdx#embeddings) and [Tier 2: Deduplication](tier2-deduplication.mdx).+Tier 2 semantic checks use an OpenAI-compatible embeddings API. Workflow context lives in [Providers & Credentials](/skills/skillevaluator/configuration#embeddings) and [Tier 2: Deduplication](/skills/skillevaluator/tier2-deduplication).| Variable | Default | Effect || --- | --- | --- |@@ -127,7 +127,7 @@ Tier 2 semantic checks use an OpenAI-compatible embeddings API. Workflow context## Validation (Tier 1)-These variables tune the static and security checks. The check-by-check guide is [Tier 1: Validation](tier1-validation.mdx).+These variables tune the static and security checks. The check-by-check guide is [Tier 1: Validation](/skills/skillevaluator/tier1-validation).| Variable | Default | Effect || --- | --- | --- |@@ -148,9 +148,9 @@ The scanner path overrides fail closed: a value that is not an absolute, executa## Live evaluation (Tier 3)These variables control where results land and how Harbor confines local mode.-The run workflow is [Tier 3: Live Evaluation](tier3-live-evaluation.mdx);+The run workflow is [Tier 3: Live Evaluation](/skills/skillevaluator/tier3-live-evaluation);execution environments are covered in-[Agents & Sandboxes](agents-and-sandboxes.mdx).+[Agents & Sandboxes](/skills/skillevaluator/agents-and-sandboxes).| Variable | Default | Effect || --- | --- | --- |@@ -167,15 +167,15 @@ execution environments are covered in| `SKILLEVALUATOR_LOCAL_STRICT_READS` | `false` | Tightens the sandbox's read-only view of the host system to a stricter path set. || `SKILLEVALUATOR_LOCAL_INHERIT_AGENT_KEYS` | `false` | When true, local-mode trials inherit the host's agent credential variables into their ambient environment. Off by default so a hostile skill command cannot read credentials — agents and verifiers receive credentials per-exec instead. || `SKILLEVALUATOR_RUNTIME_DIR` | `~/.local/share/skillevaluator/runtimes` | Where local mode looks for managed agent CLIs. Must be a dedicated subdirectory — not the home directory or one of its parents. |-| `SKILLEVALUATOR_DISABLE_CUSTOM_BACKENDS` | unset (custom backends enabled) | Set to `1`, `true`, or `yes` (case-insensitive, with surrounding whitespace ignored) to hide every installed custom execution backend and reject its mode. Other values leave discovery enabled. Built-in modes remain available. See [Custom execution backends](agents-and-sandboxes.mdx#custom-execution-backends). |+| `SKILLEVALUATOR_DISABLE_CUSTOM_BACKENDS` | unset (custom backends enabled) | Set to `1`, `true`, or `yes` (case-insensitive, with surrounding whitespace ignored) to hide every installed custom execution backend and reject its mode. Other values leave discovery enabled. Built-in modes remain available. See [Custom execution backends](/skills/skillevaluator/agents-and-sandboxes#custom-execution-backends). || `SKILLEVALUATOR_ALLOWED_MCP_HOSTS` | unset | Comma-separated hostnames approved for MCP endpoints receiving environment-variable references or authentication headers from skill-authored MCP declarations. Matching uses the whole hostname, case-insensitively; supply hostnames, not URLs or wildcard patterns. || `SKILLEVALUATOR_ALLOWED_MCP_SECRETS` | unset | Comma-separated, case-sensitive environment-variable names approved for references such as `${DEDICATED_MCP_TOKEN}` in skill-authored MCP declarations and `harbor.runtime_env`. Export the dedicated non-LLM credential on the host; literal secret values remain prohibited. Remote declarations also require the endpoint in `SKILLEVALUATOR_ALLOWED_MCP_HOSTS`. This allowlist cannot authorize operator-owned agent or provider credentials. || `SKILL_EVAL_HARBOR_PREBUILD_TASK_ENVS` | unset | Set to exactly `1` to opt in to pre-building Harbor task environment images before trials start (other truthy spellings are not honored). Opt-in because concurrent trial cleanup can remove a shared prebuilt image that sibling trials still need. || `SKILL_EVAL_JUDGE_MODEL` | falls back to `SKILL_EVAL_LLM_MODEL` | Operator-controlled model override for the Tier 3 dimension judge in standard grading. `LLM_JUDGE_MODEL` is honored as a higher-precedence spelling; when neither is set, the judge uses `SKILL_EVAL_LLM_MODEL`, then the built-in judge default. Both spellings change only the model used with the configured `SKILL_EVAL_LLM_PROVIDER`; they do not select a separate judge provider, so the model must be supported by that provider. Standard grading makes both spellings resolve to the selected override in Harbor's parent process, then forwards them through its verifier-only job layer, which takes precedence over task and step verifier declarations. Neither name is injected into the evaluated agent's initial environment; attempts to set or reference one from skill-authored `runtime_env` or native `[environment.env]` tables are rejected. Native custom-only graders keep ownership of their verifier configuration. Shared agent/verifier containers are a process boundary, not a confidentiality boundary; use a separate verifier environment when the model name itself must be hidden from untrusted agent processes. `run_config.json` records the configured judge provider, model, source, `override_applied`, reasoning effort, and catalog-verification status separately from agent models; an allowed provider fallback may use a different model for an individual judge call. When `override_applied` is false for a native Harbor task, `effective_model_source` is `native_harbor_runtime` and catalog verification is explicitly inconclusive because an authored task or step verifier declaration may supersede the recorded fallback. || `LLM_JUDGE_FALLBACK_MODELS` | unset | Comma-separated fallback models for the Tier 3 standard judges in the Harbor verifier. When the provider rejects the judge model as not found, invalid, or not accessible to the key, the verifier tries each fallback in order with the same provider. Fallback applies to the `openai`, `nv_build`, and `openai-compatible` providers. A safety refusal or an empty reply from the judge model also moves to the next fallback, on every provider. The value is forwarded only to standard verifier jobs, never to the agent environment or `custom_only` runs. || `SKILL_EVAL_LLM_REFUSAL_FALLBACK_MODEL` | `claude-opus-4-8` | Model named as the server-side refusal fallback on requests to Claude 5.5 or later models on the native Claude API; on Bedrock, the Tier 3 judge's refusal retry uses it. On Bedrock, a bare name such as `claude-opus-4-8` gets the judge model's geography prefix (`us.anthropic.`, `global.anthropic.`), and a full Bedrock ID such as `global.anthropic.claude-opus-4-8` is used as given. Set `none`, `off`, `disabled`, `false` or `0` to send no fallback, for example when the API answers HTTP 400 `is not a valid fallback target` for your model. An empty value counts as unset, and a fallback equal to the request model is never sent. Forwarded to standard verifier jobs like `LLM_JUDGE_FALLBACK_MODELS`. The reusable SkillEvaluator CI workflows do not take this variable; their provider-backed jobs keep the default. |-| `LLM_JUDGE_FALLBACK_POLICY` | `append` | Opt-in limit on which models may answer a Tier 3 standard judge. `append` keeps the built-in refusal fallback, which sends a refused Claude 5.5 or later judge to `SKILL_EVAL_LLM_REFUSAL_FALLBACK_MODEL` (default `claude-opus-4-8`, off with `none`, never sent when it equals the judge model). `replace` turns it off, so only the judge model and the `LLM_JUDGE_FALLBACK_MODELS` entries answer. Values are case-insensitive; any other value stops a run with standard grading before trials start, and the verifier fails every standard judge on it before a request is sent. Forwarded like `LLM_JUDGE_FALLBACK_MODELS`, and recorded as `judge.fallback_policy` in `run_config.json` when set. See [Judge fallback and retry policies](tier3-live-evaluation.mdx#judge-fallback-and-retry-policies). |-| `LLM_JUDGE_RETRY_POLICY` | `legacy` | Opt-in retry policy for the Tier 3 standard judges. `strict` tries at most three models (the judge model, then the first two `LLM_JUDGE_FALLBACK_MODELS` entries) with at most two requests each within the per-judge time budget, moves on only when a model is unavailable, ends the judge on a refusal or a quota, budget, safety, or security error, and scores only a first completed reply that is exactly one JSON object. `legacy` keeps the standard retries, fallbacks, format retry, and salvage. Values are case-insensitive; any other value stops a run with standard grading before trials start, and the verifier fails every standard judge on it before a request is sent. Forwarded like `LLM_JUDGE_FALLBACK_MODELS`, and recorded as `judge.retry_policy` in `run_config.json` when set. See [Judge fallback and retry policies](tier3-live-evaluation.mdx#judge-fallback-and-retry-policies). |+| `LLM_JUDGE_FALLBACK_POLICY` | `append` | Opt-in limit on which models may answer a Tier 3 standard judge. `append` keeps the built-in refusal fallback, which sends a refused Claude 5.5 or later judge to `SKILL_EVAL_LLM_REFUSAL_FALLBACK_MODEL` (default `claude-opus-4-8`, off with `none`, never sent when it equals the judge model). `replace` turns it off, so only the judge model and the `LLM_JUDGE_FALLBACK_MODELS` entries answer. Values are case-insensitive; any other value stops a run with standard grading before trials start, and the verifier fails every standard judge on it before a request is sent. Forwarded like `LLM_JUDGE_FALLBACK_MODELS`, and recorded as `judge.fallback_policy` in `run_config.json` when set. See [Judge fallback and retry policies](/skills/skillevaluator/tier3-live-evaluation#judge-fallback-and-retry-policies). |+| `LLM_JUDGE_RETRY_POLICY` | `legacy` | Opt-in retry policy for the Tier 3 standard judges. `strict` tries at most three models (the judge model, then the first two `LLM_JUDGE_FALLBACK_MODELS` entries) with at most two requests each within the per-judge time budget, moves on only when a model is unavailable, ends the judge on a refusal or a quota, budget, safety, or security error, and scores only a first completed reply that is exactly one JSON object. `legacy` keeps the standard retries, fallbacks, format retry, and salvage. Values are case-insensitive; any other value stops a run with standard grading before trials start, and the verifier fails every standard judge on it before a request is sent. Forwarded like `LLM_JUDGE_FALLBACK_MODELS`, and recorded as `judge.retry_policy` in `run_config.json` when set. See [Judge fallback and retry policies](/skills/skillevaluator/tier3-live-evaluation#judge-fallback-and-retry-policies). || `SKILLEVALUATOR_HARBOR_DIAGNOSTIC_ARTIFACT_MAX_BYTES` | `5242880` (5 MiB) | Size cap per diagnostic artifact copied from a Harbor trial into the results directory. Values are bounded to 0–67108864 bytes (64 MiB); `0` skips non-empty artifacts. Oversized artifacts are skipped and recorded in the artifact manifest. || `SKILLEVALUATOR_GKE_CLUSTER`, `SKILLEVALUATOR_GKE_REGION`, `SKILLEVALUATOR_GKE_NAMESPACE`, `SKILLEVALUATOR_GKE_REGISTRY_LOCATION`, `SKILLEVALUATOR_GKE_REGISTRY_NAME` | — | Target cluster, region, namespace, and Artifact Registry coordinates for `--env-mode gke` when not supplied via `--ek`. || `SKILLEVALUATOR_GKE_ALLOW_WORKLOAD_IDENTITY` | `false` | Opt in (`1`, `true`, `yes`) to single-pod GKE Workload Identity (`CLAUDE_CODE_USE_VERTEX=1` or a GCP-bound Kubernetes ServiceAccount). Restricted to trusted skills because the pod ServiceAccount and GCE/GKE metadata server are shared with evaluated skill commands. |@@ -194,7 +194,7 @@ directory with `.env` loading and Harbor telemetry disabled:- An evaluation run passes Harbor only an allowlisted environment: basicprocess, locale, and temporary-directory variables such as `PATH`, `HOME`,`LANG`, and `TMPDIR`; proxy and CA-bundle variables; the selected backend's- own variables (see [Agents & Sandboxes](agents-and-sandboxes.mdx)); the+ own variables (see [Agents & Sandboxes](/skills/skillevaluator/agents-and-sandboxes)); theevaluator provider's settings and credential, plus the `AWS_*` chain for`bedrock`; each selected agent's credentials; validated `runtime_env`values; and the Tier 3 judge, retry, and evidence-budget variables on this@@ -206,7 +206,7 @@ directory with `.env` loading and Harbor telemetry disabled:so Docker Desktop can discover Compose. `PYTHONUTF8` and`PYTHONIOENCODING` are also forwarded when set; use `PYTHONUTF8=1` and`PYTHONIOENCODING=utf-8` for a non-UTF-8 locale. See the- [PowerShell setup](agents-and-sandboxes.mdx#windows-hosts).+ [PowerShell setup](/skills/skillevaluator/agents-and-sandboxes#windows-hosts).- Setting `HARBOR_TELEMETRY` or `PYTHON_DOTENV_DISABLED` yourself has noeffect; SkillEvaluator always overrides both.- Tier 3 needs python-dotenv 1.2.0 or newer, which the `tier3` extra installs.@@ -220,7 +220,7 @@ to standard Tier 3 judge prompts. It is off by default; `true`, `yes`, and `on`also enable it (case-insensitive). This operator-owned setting is forwarded onlyto standard verifier jobs, never to agent environments or `custom_only` jobs.When configured, its value is recorded as `judge.character_facts` in-`run_config.json`. See [character-sensitive judging](tier3-live-evaluation.mdx#character-sensitive-judging)+`run_config.json`. See [character-sensitive judging](/skills/skillevaluator/tier3-live-evaluation#character-sensitive-judging)for scope, limits, and the effect on the goal-accuracy route.Before the Harbor verifier calls the `accuracy`, `goal_accuracy`, and@@ -241,7 +241,7 @@ Setting `SKILL_EVAL_BEHAVIOR_CHECK_BUDGET` below `SKILL_EVAL_BEHAVIOR_FINAL_RESPplus 1600 is honoured, but logs a warning: the final response will not fitbeside any tool history, and `behavior_check` then reports incomplete evidenceinstead of a score (see-[behavior evidence completeness](tier3-live-evaluation.mdx#evidence-budgets)).+[behavior evidence completeness](/skills/skillevaluator/tier3-live-evaluation#evidence-budgets)).Lower the final-response limit alongside the budget to cap the judge promptwithout losing the answer.@@ -262,7 +262,7 @@ In standard grading, a required judge transport, authentication, timeout, HTTP,## In-container grader contract-The `HARBOR_*` variables are read by your custom grader *inside* the task container — you never set them yourself, and each has a fixed in-container default. The full reward contract, with defaults and examples, lives in [Custom Graders & Tasks](custom-graders.mdx).+The `HARBOR_*` variables are read by your custom grader *inside* the task container — you never set them yourself, and each has a fixed in-container default. The full reward contract, with defaults and examples, lives in [Custom Graders & Tasks](/skills/skillevaluator/custom-graders).| Variable | What it carries || --- | --- |@@ -277,14 +277,14 @@ The `HARBOR_*` variables are read by your custom grader *inside* the task contai## Set at runtime, not committed<Warning>-Keep credentials out of skill repositories and CI configuration files. Export provider keys as masked secrets at runtime. Agent credentials for Tier 3 come only from the host environment — `evals/config.yml` `runtime_env` carries non-credential values, and any entry that names or references an operator-owned credential (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `NVIDIA_API_KEY`, their base-URL variables, `CLAUDE_CODE_USE_BEDROCK`, and the `AWS_*` credential variables) is rejected at run time, as are names that control the host process (`PATH`, `DOCKER_*`, proxy variables, `SKILL_EVAL_*`, `SKILLEVALUATOR_*`, and similar). See the blocked-names table in [Providers & Credentials](configuration.mdx#tier-3-credentials-two-roles). With NVIDIA Build, one exported `NVIDIA_API_KEY` covers both the evaluator and the agents in the `docker`, `kata`, and `local` env-modes.+Keep credentials out of skill repositories and CI configuration files. Export provider keys as masked secrets at runtime. Agent credentials for Tier 3 come only from the host environment — `evals/config.yml` `runtime_env` carries non-credential values, and any entry that names or references an operator-owned credential (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `NVIDIA_API_KEY`, their base-URL variables, `CLAUDE_CODE_USE_BEDROCK`, and the `AWS_*` credential variables) is rejected at run time, as are names that control the host process (`PATH`, `DOCKER_*`, proxy variables, `SKILL_EVAL_*`, `SKILLEVALUATOR_*`, and similar). See the blocked-names table in [Providers & Credentials](/skills/skillevaluator/configuration#tier-3-credentials-two-roles). With NVIDIA Build, one exported `NVIDIA_API_KEY` covers both the evaluator and the agents in the `docker`, `kata`, and `local` env-modes.</Warning>## See also-- [Providers & Credentials](configuration.mdx) — the setup walkthrough behind the provider variables-- [CLI Reference](cli-reference.mdx) — the per-run flags that override many of these variables-- [Gate Your CI](ci-integration.mdx) — supplying these variables as masked CI secrets+- [Providers & Credentials](/skills/skillevaluator/configuration) — the setup walkthrough behind the provider variables+- [CLI Reference](/skills/skillevaluator/cli-reference) — the per-run flags that override many of these variables+- [Gate Your CI](/skills/skillevaluator/ci-integration) — supplying these variables as masked CI secrets## GitLab all-tier admissiondiff --git a/docs/eval-datasets.mdx b/docs/eval-datasets.mdxindex 61eae6a3..99e857b2 100644--- a/docs/eval-datasets.mdx+++ b/docs/eval-datasets.mdx@@ -685,11 +685,11 @@ Passing checks and warnings are not listed as source errors.## Next steps<CardGroup cols={2}>- <Card title="Run a live evaluation" icon="rocket" href="tier3-live-evaluation.mdx">+ <Card title="Run a live evaluation" icon="rocket" href="/skills/skillevaluator/tier3-live-evaluation">Point `tier3 evaluate` at your skill and measure Skill Lift with thedataset you just authored.</Card>- <Card title="Custom graders and tasks" icon="flask" href="custom-graders.mdx">+ <Card title="Custom graders and tasks" icon="flask" href="/skills/skillevaluator/custom-graders">When assertions aren't enough, bring your own grader or a complete Harbortask.</Card>diff --git a/docs/index.mdx b/docs/index.mdxindex 075d42b7..eace890a 100644--- a/docs/index.mdx+++ b/docs/index.mdx@@ -6,7 +6,7 @@ layout: overviewSkillEvaluator is an open-source, multi-tier framework for evaluating AI agent artifacts, starting with agent skills: deterministic quality gates, semantic overlap detection, synthetic eval dataset generation, and live agent evaluation.-Agent skills are folders of instructions and supporting files that extend AI agents, as defined by the [Agent Skills specification](https://agentskills.io/). SkillEvaluator is part of the [NVIDIA Verified Skills pipeline](https://docs.nvidia.com/skills/), published from the [NVIDIA/skills catalog](https://github.com/NVIDIA/skills). Tier 1 integrates [SkillSpector](https://github.com/NVIDIA/SkillSpector) for specialized security scanning; see [Installation](installation.mdx#system-tools) for the security setup.+Agent skills are folders of instructions and supporting files that extend AI agents, as defined by the [Agent Skills specification](https://agentskills.io/). SkillEvaluator is part of the [NVIDIA Verified Skills pipeline](https://docs.nvidia.com/skills/), published from the [NVIDIA/skills catalog](https://github.com/NVIDIA/skills). Tier 1 integrates [SkillSpector](https://github.com/NVIDIA/SkillSpector) for specialized security scanning; see [Installation](/skills/skillevaluator/installation#system-tools) for the security setup.<Note>SkillEvaluator's support level is **Experimental**: community-supported on a best-effort basis through [GitHub Issues](https://github.com/NVIDIA/SkillEvaluator/issues), with no SLA. See [SUPPORT.md](https://github.com/NVIDIA/SkillEvaluator/blob/main/SUPPORT.md).@@ -19,7 +19,7 @@ Agent skills are folders of instructions and supporting files that extend AI ageChoose a deterministic quality check or an LLM-backed rubric review. Both routesuse the bundled `text-analyzer` skill, so you can run them before preparing yourown skill. Neither route needs Docker or an agent runtime. Set up your chosen-agent and sandbox when you move on to [live Tier 3 evaluation](tier3-live-evaluation.mdx).+agent and sandbox when you move on to [live Tier 3 evaluation](/skills/skillevaluator/tier3-live-evaluation).### Run without an LLM@@ -35,11 +35,11 @@ uv run --python 3.13 skillevaluator quality-check src/skillevaluator/tier3/referRead the score, grade, and findings in the terminal, and open`reports/first-quality/skillevaluator-quality.json` for the saved result.This checks documentation quality; measuring the skill's effect on an agent-requires a [live evaluation](tier3-live-evaluation.mdx).+requires a [live evaluation](/skills/skillevaluator/tier3-live-evaluation).-The [keyless Quickstart](quickstart.mdx#run-without-an-llm) explains the result+The [keyless Quickstart](/skills/skillevaluator/quickstart#run-without-an-llm) explains the resultand how to check your own skill. The full static security suite has additional-[scanner requirements](installation.mdx#system-tools), even though it also+[scanner requirements](/skills/skillevaluator/installation#system-tools), even though it alsoruns without an LLM or API key.### Use NVIDIA Build@@ -54,7 +54,7 @@ uv run --python 3.13 --extra llm skillevaluator rubric-eval src/skillevaluator/t```The `llm` extra installs the provider clients for this model call. The-[NVIDIA Build Quickstart](quickstart.mdx#use-nvidia-build) covers access,+[NVIDIA Build Quickstart](/skills/skillevaluator/quickstart#use-nvidia-build) covers access,outputs, and the distinction between rubric scores and observed agent behavior.## The three tiers@@ -62,13 +62,13 @@ outputs, and the distinction between rubric scores and observed agent behavior.Each tier answers one question about your skill.<CardGroup cols={3}>- <Card title="Tier 1: Validation" href="tier1-validation.mdx">+ <Card title="Tier 1: Validation" href="/skills/skillevaluator/tier1-validation">**Is this skill safe and well-formed?** Deterministic schema, quality, security, PII, license, and script checks that run offline — plus optional LLM-as-judge scoring.</Card>- <Card title="Tier 2: Deduplication" href="tier2-deduplication.mdx">+ <Card title="Tier 2: Deduplication" href="/skills/skillevaluator/tier2-deduplication">**Does it overlap with what exists?** Embedding similarity finds repeated guidance inside one skill and overlapping skills across a collection or catalog.</Card>- <Card title="Tier 3: Live Evaluation" href="tier3-live-evaluation.mdx">+ <Card title="Tier 3: Live Evaluation" href="/skills/skillevaluator/tier3-live-evaluation">**Does it actually help the agent?** A real agent runs generated tasks with and without the skill, inside a sandbox, and the difference is measured.</Card></CardGroup>@@ -77,9 +77,9 @@ Every tier is an independent entry point — run any command directly; nothing r| Tier | Representative commands | Requires || --- | --- | --- |-| [Tier 1](tier1-validation.mdx) | `tier1 PATH`, `quality-check`, `security-scan`, `pii-scan`, `lint-scripts`, `rubric-eval` | No API key for deterministic checks; external Semgrep, SkillSpector, and Gitleaks executables for full scanner coverage; a provider key for LLM checks |-| [Tier 2](tier2-deduplication.mdx) | `context-optimization-check`, `similarity-check` | An embeddings provider; intra-skill analysis also needs a chat LLM — local OpenAI-compatible endpoints work |-| [Tier 3](tier3-live-evaluation.mdx) | `create-eval-dataset`, `tier3 evaluate`, `compare` | No credential for keyless templates and report inspection; a provider key for LLM generation and grading; live evaluation also needs the agent CLI with its credential and a Docker, local OS, or cloud sandbox |+| [Tier 1](/skills/skillevaluator/tier1-validation) | `tier1 PATH`, `quality-check`, `security-scan`, `pii-scan`, `lint-scripts`, `rubric-eval` | No API key for deterministic checks; external Semgrep, SkillSpector, and Gitleaks executables for full scanner coverage; a provider key for LLM checks |+| [Tier 2](/skills/skillevaluator/tier2-deduplication) | `context-optimization-check`, `similarity-check` | An embeddings provider; intra-skill analysis also needs a chat LLM — local OpenAI-compatible endpoints work |+| [Tier 3](/skills/skillevaluator/tier3-live-evaluation) | `create-eval-dataset`, `tier3 evaluate`, `compare` | No credential for keyless templates and report inspection; a provider key for LLM generation and grading; live evaluation also needs the agent CLI with its credential and a Docker, local OS, or cloud sandbox |## How a run flows@@ -89,7 +89,7 @@ Every tier is an independent entry point — run any command directly; nothing r`skillevaluator validate ./my-skill` runs all three tiers by default, withTier 3 autopilot reusing an existing evaluation source or creating one starter-case when none exists. Tier 1's deterministic gates run offline. Tier 2 and the optional Tier 1 LLM checks need a provider key, and Tier 3 adds the agent credential and a sandbox. Every run ends in [reports](reports.mdx) — CLI, JSON, HTML, or Markdown.+case when none exists. Tier 1's deterministic gates run offline. Tier 2 and the optional Tier 1 LLM checks need a provider key, and Tier 3 adds the agent credential and a sandbox. Every run ends in [reports](/skills/skillevaluator/reports) — CLI, JSON, HTML, or Markdown.## What Tier 3 measures@@ -97,24 +97,24 @@ case when none exists. Tier 1's deterministic gates run offline. Tier 2 and the- **Skill Lift** — the measured difference between the with-skill and without-skill runs; the number that says whether the skill earns its place.- **pass@k** — multi-attempt reliability, reported separately from the dimension scores.-How to read all three, and the on-disk results contract, live in [Reports & Results](reports.mdx).+How to read all three, and the on-disk results contract, live in [Reports & Results](/skills/skillevaluator/reports).## Explore the docs<CardGroup cols={2}>- <Card title="Quickstart" icon="rocket" href="quickstart.mdx">+ <Card title="Quickstart" icon="rocket" href="/skills/skillevaluator/quickstart">Run a bundled quality check without an LLM, or use NVIDIA Build for a rubric review.</Card>- <Card title="Evaluate from Your Agent" icon="robot" href="agent-skill.mdx">+ <Card title="Evaluate from Your Agent" icon="robot" href="/skills/skillevaluator/agent-skill">Install the SkillEvaluator skill in Codex, Claude Code, OpenCode, or OpenClaw and evaluate skills by asking.</Card>- <Card title="Gate Your CI" icon="shield" href="ci-integration.mdx">+ <Card title="Gate Your CI" icon="shield" href="/skills/skillevaluator/ci-integration">Make SkillEvaluator a merge gate — exit codes, a GitHub Actions recipe, and progressive adoption.</Card>- <Card title="CLI Reference" icon="terminal" href="cli-reference.mdx">+ <Card title="CLI Reference" icon="terminal" href="/skills/skillevaluator/cli-reference">Every command, every flag, exact defaults.</Card>- <Card title="Contributing" icon="book" href="developer-guide.mdx">+ <Card title="Contributing" icon="book" href="/skills/skillevaluator/developer-guide">Dev environment, tests, docs workflow, and how to submit a change.</Card></CardGroup>diff --git a/docs/installation.mdx b/docs/installation.mdxindex 83eaf89d..13f007b5 100644--- a/docs/installation.mdx+++ b/docs/installation.mdx@@ -6,12 +6,12 @@ description: "Install SkillEvaluator with uv, from source, or in Docker, and choStart with the base package for deterministic quality checks, then add only theextras needed for your next command. No API key is needed to install or to rundeterministic Tier 1 checks. LLM-backed features use a configured provider-([Providers & Credentials](configuration.mdx)); live Tier 3 evaluation adds an+([Providers & Credentials](/skills/skillevaluator/configuration)); live Tier 3 evaluation adds anagent runtime and a sandbox.For a complete first run against a bundled skill, use the-[keyless Quickstart](quickstart.mdx#run-without-an-llm), or add the `llm` extra-for the [NVIDIA Build route](quickstart.mdx#use-nvidia-build). Those routes use a+[keyless Quickstart](/skills/skillevaluator/quickstart#run-without-an-llm), or add the `llm` extra+for the [NVIDIA Build route](/skills/skillevaluator/quickstart#use-nvidia-build). Those routes use asource checkout and `uv run`; the standalone install below is convenient whenyou already have your own skill directory.@@ -26,7 +26,7 @@ This project will download and install additional third-party open source softwa- No API key is needed for deterministic Tier 1 checks. The base packageincludes Bandit and the uv executable; Tier 1 provisions SkillSpector andGitleaks itself, and Semgrep is optional (see [System tools](#system-tools)). LLM-backed checks and Tiers 2–3 use a configured provider — see- [Providers & Credentials](configuration.mdx).+ [Providers & Credentials](/skills/skillevaluator/configuration).## Install@@ -50,7 +50,7 @@ Add the `llm` extra when you want to make your first rubric-evaluation call:uv tool install --python 3.13 --reinstall "skillevaluator[llm] @ git+https://github.com/NVIDIA/SkillEvaluator.git"```-Then follow the [NVIDIA Build setup](quickstart.mdx#use-nvidia-build), using+Then follow the [NVIDIA Build setup](/skills/skillevaluator/quickstart#use-nvidia-build), using`skillevaluator rubric-eval ./my-skill` for a standalone installation. Providerclients install with the extra; an API key is still required for the model call.@@ -83,7 +83,7 @@ skillevaluator --version</Tab><Tab title="From source" language="source">-The [Quickstart](quickstart.mdx) uses `uv run --python 3.13` for the base quality+The [Quickstart](/skills/skillevaluator/quickstart) uses `uv run --python 3.13` for the base qualitycheck and adds `--extra llm` for the NVIDIA Build rubric call. This installsonly the dependencies needed for each route.@@ -178,7 +178,7 @@ and environment backends.| `dev` | Build, test, coverage, and formatting tools for contributors |<Tip>-`[all]` installs the evaluator extras (`tier2`, `tier3`, `security`); add `dashboard` with `[all,dashboard]`. External scanner tools, Docker, agent CLIs, and the Harbor extras for managed sandbox backends (such as `harbor[modal]==0.25.0`) stay separate no matter which extra you pick — see [Agents & Sandboxes](agents-and-sandboxes.mdx#where-trials-run).+`[all]` installs the evaluator extras (`tier2`, `tier3`, `security`); add `dashboard` with `[all,dashboard]`. External scanner tools, Docker, agent CLIs, and the Harbor extras for managed sandbox backends (such as `harbor[modal]==0.25.0`) stay separate no matter which extra you pick — see [Agents & Sandboxes](/skills/skillevaluator/agents-and-sandboxes#where-trials-run).</Tip>## System tools@@ -198,7 +198,7 @@ uv tool install git+https://github.com/NVIDIA/[email protected]brew install gitleaks # or a binary from https://github.com/gitleaks/gitleaks/releases```-Docker is required only for Tier 3 Docker-mode evaluation. Agent CLIs and their credentials are user-supplied — see [Agents & Sandboxes](agents-and-sandboxes.mdx).+Docker is required only for Tier 3 Docker-mode evaluation. Agent CLIs and their credentials are user-supplied — see [Agents & Sandboxes](/skills/skillevaluator/agents-and-sandboxes).`skillevaluator health-check --tier 1` reports the path and version of each installed scanner and exits non-zero when one is missing, misconfigured, or older than the supported release. Add `--checks security,code-integrity,dependency` to include pip-audit.</Note>@@ -216,7 +216,7 @@ skillevaluator --versionskillevaluator health-check```-`--version` confirms the CLI is on PATH — that is the whole smoke test for a Tier 1-only install. `health-check` goes one step further: a quick readiness check of the CLI and the selected live-eval backend (the agent follows the configured provider; the environment defaults to `docker`), useful once you plan to run Tier 3. On an install without the `tier3` extra or a configured provider it lists the missing pieces and exits non-zero — expected for a Tier 1-only setup, not a broken install. For a deeper diagnosis, `doctor` covers the same ground; add `doctor --verify-models` to make a live credential-bearing catalog request for each resolved agent model. A public or otherwise non-authoritative result is reported as a warning because catalog access alone does not prove inference access. See the [CLI Reference](cli-reference.mdx) for both commands.+`--version` confirms the CLI is on PATH — that is the whole smoke test for a Tier 1-only install. `health-check` goes one step further: a quick readiness check of the CLI and the selected live-eval backend (the agent follows the configured provider; the environment defaults to `docker`), useful once you plan to run Tier 3. On an install without the `tier3` extra or a configured provider it lists the missing pieces and exits non-zero — expected for a Tier 1-only setup, not a broken install. For a deeper diagnosis, `doctor` covers the same ground; add `doctor --verify-models` to make a live credential-bearing catalog request for each resolved agent model. A public or otherwise non-authoritative result is reported as a warning because catalog access alone does not prove inference access. See the [CLI Reference](/skills/skillevaluator/cli-reference) for both commands.## Troubleshooting@@ -232,10 +232,10 @@ A required scanner could not run. Check that GitHub and PyPI are reachable (Skil## Next steps<CardGroup cols={2}>- <Card title="Quickstart" icon="rocket" href="quickstart.mdx">+ <Card title="Quickstart" icon="rocket" href="/skills/skillevaluator/quickstart">Check a bundled skill without an LLM, or use NVIDIA Build for a rubric review.</Card>- <Card title="Providers & Credentials" icon="shield" href="configuration.mdx">+ <Card title="Providers & Credentials" icon="shield" href="/skills/skillevaluator/configuration">Wire up an LLM provider for Tier 2, Tier 3, and LLM-backed checks.</Card></CardGroup>diff --git a/docs/performance-dashboard.mdx b/docs/performance-dashboard.mdxindex 368ee688..2a51ea6a 100644--- a/docs/performance-dashboard.mdx+++ b/docs/performance-dashboard.mdx@@ -56,7 +56,7 @@ excluded from lift, so it can differ from subtracting the displayed arm scores.Pass-rate and resource differences can remain available when score lift isunavailable. If every positive with-skill trial has confirmed zero activation,the comparison is **inconclusive** and all differences stay unavailable. See-[Skill Lift](reports.mdx#skill-lift) for these eligibility rules.+[Skill Lift](/skills/skillevaluator/reports#skill-lift) for these eligibility rules.When a pair's differences stay blank, the **Comparison reason** column says why.Repeated attempts pair by case, as in the HTML report: each arm must record the@@ -85,7 +85,7 @@ Missing measurements stay unavailable, and cost is shown only when recorded inUSD. Canonical Tier 3 JSON records each single-step trial's agent execution time(`agent_seconds`) when Harbor's timestamps carry a timezone. Timestamps without atimezone are treated as unavailable, as in the HTML report's-[measured agent resources](reports.mdx#measured-agent-resources). Agent time excludes+[measured agent resources](/skills/skillevaluator/reports#measured-agent-resources). Agent time excludesenvironment setup and verification. Canonical JSON never records USD cost; openthe retained run directory to show Harbor's recorded `agent_result.cost_usd` andmulti-step totals. For retained runs whose trajectory lacks token totals ordiff --git a/docs/quickstart.mdx b/docs/quickstart.mdxindex 1bc47be0..bbaa1f00 100644--- a/docs/quickstart.mdx+++ b/docs/quickstart.mdx@@ -13,7 +13,7 @@ quality check, or add NVIDIA Build to get feedback from a model.For an executable terminal replay with checked outputs, an editable task, adeliberate regression and report interpretation, use the-[reproducible starter demo](starter-demo.mdx). It uses the same bundled skill.+[reproducible starter demo](/skills/skillevaluator/starter-demo). It uses the same bundled skill.## Get the example@@ -54,7 +54,7 @@ This checks static quality across correctness, discoverability, reliability,and efficiency. It does not run the security scanners or measure an agent'sperformance. The report's publication status remains `INCOMPLETE` because aquality score alone does not supply all required evaluation evidence. See-[Reports & Results](reports.mdx) for the full output contract.+[Reports & Results](/skills/skillevaluator/reports) for the full output contract.## Use NVIDIA Build@@ -96,7 +96,7 @@ reports/first-build/skillevaluator-rubric.jsonScores can vary between model calls. This evaluates the skill's instructions;it does not run an agent with the skill. For model overrides or another-provider, see [Providers & Credentials](configuration.mdx).+provider, see [Providers & Credentials](/skills/skillevaluator/configuration).## Where to go next@@ -114,7 +114,7 @@ uv run --python 3.13 skillevaluator quality-check ./my-skill \Full static Tier 1 coverage needs the separateSemgrep, SkillSpector, and Gitleaks executables. Follow-[scanner setup](installation.mdx#system-tools), then run:+[scanner setup](/skills/skillevaluator/installation#system-tools), then run:```bash title="Run the static Tier 1 suite"uv run --python 3.13 skillevaluator tier1 ./my-skill --no-llm@@ -122,18 +122,18 @@ uv run --python 3.13 skillevaluator tier1 ./my-skill --no-llm`--no-llm` disables model-backed stages; it does not remove scannerrequirements. Missing required scanner evidence leaves validation `INCOMPLETE`-and exits nonzero. See [Tier 1: Validation](tier1-validation.mdx) for the+and exits nonzero. See [Tier 1: Validation](/skills/skillevaluator/tier1-validation) for thechecks and exit codes.### Add deduplication or live evaluation| Your goal | Additional setup | Guide || --- | --- | --- |-| Find repeated or overlapping guidance | The `tier2` extra, embeddings, and chat for intra-skill analysis | [Tier 2: Deduplication](tier2-deduplication.mdx) |-| Measure whether a skill improves an agent | The `tier3` extra, evaluator and agent credentials, a supported agent, and Docker for the default sandbox | [Tier 3: Live Evaluation](tier3-live-evaluation.mdx) |-| Run the combined pipeline or gate CI | The extras, credentials, scanners, and runtime for the selected tiers | [CLI Reference](cli-reference.mdx#validate) and [Gate Your CI](ci-integration.mdx) |+| Find repeated or overlapping guidance | The `tier2` extra, embeddings, and chat for intra-skill analysis | [Tier 2: Deduplication](/skills/skillevaluator/tier2-deduplication) |+| Measure whether a skill improves an agent | The `tier3` extra, evaluator and agent credentials, a supported agent, and Docker for the default sandbox | [Tier 3: Live Evaluation](/skills/skillevaluator/tier3-live-evaluation) |+| Run the combined pipeline or gate CI | The extras, credentials, scanners, and runtime for the selected tiers | [CLI Reference](/skills/skillevaluator/cli-reference#validate) and [Gate Your CI](/skills/skillevaluator/ci-integration) |-The [live evaluation guide](tier3-live-evaluation.mdx#before-you-start) covers+The [live evaluation guide](/skills/skillevaluator/tier3-live-evaluation#before-you-start) coversruntime readiness and the first with-skill and without-skill comparison.-[Installation](installation.mdx#choosing-extras) explains how to add each extra+[Installation](/skills/skillevaluator/installation#choosing-extras) explains how to add each extrato your checkout commands with `uv run --python 3.13 --extra EXTRA`.diff --git a/docs/reports.mdx b/docs/reports.mdxindex c8e7de74..a93ef416 100644--- a/docs/reports.mdx+++ b/docs/reports.mdx@@ -57,7 +57,7 @@ stops before any check and asks for an explicit `-o`/`--output-dir`.Ownership is tied to a key stored in your home directory (see`SKILLEVALUATOR_OUTPUT_PROVENANCE_KEY_FILE` in-[Environment Variables](environment-variables.mdx)). Report folders written+[Environment Variables](/skills/skillevaluator/environment-variables)). Report folders writtenwith another key, for example by another account or by a container whose homedirectory was discarded, count as unowned. Pass `-o` in those setups.diff --git a/docs/starter-demo.mdx b/docs/starter-demo.mdxindex 0a263410..e867425b 100644--- a/docs/starter-demo.mdx+++ b/docs/starter-demo.mdx@@ -27,7 +27,7 @@ the public release wheel, not an editable development install. The 0.8.0 wheelis available on [GitHub Releases](https://github.com/NVIDIA/SkillEvaluator/releases/tag/v0.8.0);do not assume the same version is already on PyPI. Installation needsinternet access. The demo then runs locally. For standalone pip installations-and supported Python versions, see [Installation](installation.mdx).+and supported Python versions, see [Installation](/skills/skillevaluator/installation).The runner prints each command and its real terminal output. It copies theinstalled skill into `reports/starter/text-analyzer`, stages the public input@@ -69,7 +69,7 @@ precedence over missing evidence. Do not interpret incomplete evidence as azero score, a pass, or proof of a skill defect. Fix execution/setup problems andrerun before drawing performance conclusions. For live agent results, comparewith-skill and without-skill outcomes, inspect failed attempts and coverage, and-use the verdict and completion fields together; see [Reports & Results](reports.mdx).+use the verdict and completion fields together; see [Reports & Results](/skills/skillevaluator/reports).### Replay a single command manually@@ -87,8 +87,8 @@ prebuilt report. The usual quality threshold is 70; this demo deliberately uses## Add model feedback or a live agentFor fresh model-backed instruction feedback on this same skill, configure an-account and key using [NVIDIA Build](quickstart.mdx#use-nvidia-build) or another-[public provider](configuration.mdx). Never put a key in a report or fixture.+account and key using [NVIDIA Build](/skills/skillevaluator/quickstart#use-nvidia-build) or another+[public provider](/skills/skillevaluator/configuration). Never put a key in a report or fixture.Then run:```bash@@ -99,7 +99,7 @@ uv run --isolated --no-project --python 3.13 --with "skillevaluator[llm] @ httpsModel catalogs and account access change. A visible catalog entry does notguarantee inference access. If the default model returns HTTP 404 or 410, selecta model available to your account with `SKILL_EVAL_LLM_MODEL` as described in-[configuration](configuration.mdx). Authentication errors, rate limits, and+[configuration](/skills/skillevaluator/configuration). Authentication errors, rate limits, andunavailable models are not low rubric scores. Inspect the JSON findings for`llm_unavailable`; do not present a fallback response as a completed evaluation.@@ -114,7 +114,7 @@ supported model rather than interpreting it as a completed low-scoring run.Rubric feedback is not agent uplift. For the paired agent experiment, keep thesame copied task and input and follow the-[live evaluation prerequisites](tier3-live-evaluation.mdx#before-you-start).+[live evaluation prerequisites](/skills/skillevaluator/tier3-live-evaluation#before-you-start).After installing the `tier3` extra, configuring evaluator/agent credentials andstarting Docker, run:@@ -127,7 +127,7 @@ The authored task is `evals/evals.json`; `evals/files/input.txt` is staged intothe sandbox at `/workspace/input/input.txt`. Read both before paying for agentruns. One tiny task is a plumbing example, not representative coverage orindependent validation of a grader. Model and sandbox usage can incur charges.-Read [Eval Datasets](eval-datasets.mdx) for additional cases, grading and policy,+Read [Eval Datasets](/skills/skillevaluator/eval-datasets) for additional cases, grading and policy,and retain all completed attempts and errors rather than publishing only a score.## Adapt the example@@ -162,15 +162,15 @@ self-checks are not a substitute for feedback from a first-time human user.| Symptom | Action || --- | --- |-| Python or dependencies cannot download | Check internet/proxy access and uv installation; see [installation](installation.mdx). No key is needed for the keyless route. |+| Python or dependencies cannot download | Check internet/proxy access and uv installation; see [installation](/skills/skillevaluator/installation). No key is needed for the keyless route. || Output directory already exists | Keep it as evidence; use a new `--output` path. Nothing is overwritten. || A checked score or report field differs | Read the retained JSON/text, confirm `receipt.json` if present and the installed version; do not blindly update expectations. A failed demo does not write a success receipt. || Analysis count differs | Check file contents, encoding and newlines; the input is UTF-8 plain text with a final newline. |-| Scanner or sandbox is missing | The keyless demo does not need either. Follow [scanner setup](installation.mdx#system-tools) or [agent setup](agents-and-sandboxes.mdx) before adding those stages. |+| Scanner or sandbox is missing | The keyless demo does not need either. Follow [scanner setup](/skills/skillevaluator/installation#system-tools) or [agent setup](/skills/skillevaluator/agents-and-sandboxes) before adding those stages. || Template reference asks you to sign in | Use the [public template reference](https://github.com/NVIDIA/SkillEvaluator/blob/main/ci/templates/README.md) and [versioned bundles](https://github.com/NVIDIA/SkillEvaluator/tree/main/ci/bundles/0.8.0), not a private mirror. Check bundle files against `SHA256SUMS.txt`. |The public template reference and versioned downloadable bundles were consolidatedin [the public template update](https://github.com/NVIDIA/SkillEvaluator/pull/328).They require no private account. For repository automation, continue with-[Repository Integration](repository-integration.mdx); for improvements to this-starter, see [Contributing](developer-guide.mdx).+[Repository Integration](/skills/skillevaluator/repository-integration); for improvements to this+starter, see [Contributing](/skills/skillevaluator/developer-guide).diff --git a/docs/tier1-validation.mdx b/docs/tier1-validation.mdxindex 5508da2c..68f0fb20 100644--- a/docs/tier1-validation.mdx+++ b/docs/tier1-validation.mdx@@ -16,9 +16,9 @@ runs rubric judging, LLM security analysis, and finding verification. Without aprovider, those LLM stages are explicitly reported as skipped. Use `--no-llm`for static checks only, or `--llm` to require a provider instead of allowing akeyless run. Full scanner coverage still requires the external tools listed in-[Installation](installation.mdx#system-tools).+[Installation](/skills/skillevaluator/installation#system-tools).-The two-variable [provider setup](configuration.mdx) is enough to configure the+The two-variable [provider setup](/skills/skillevaluator/configuration) is enough to configure theLLM stages for NVIDIA Build or OpenAI. Calling `skillevaluator tier1` without apath shows help; expert subcommands remain available below.@@ -36,8 +36,8 @@ when it changes a skill. Eligible pushes and manual runs on the default branchrun them for the whole collection.Existing released Tier 1-only consumers keep their current workflow and-bundled image pins. See [GitHub Actions](github-actions.mdx) or-[GitLab CI/CD](gitlab-ci.mdx) to choose either path.+bundled image pins. See [GitHub Actions](/skills/skillevaluator/github-actions) or+[GitLab CI/CD](/skills/skillevaluator/gitlab-ci) to choose either path.## What Tier 1 checks@@ -68,7 +68,7 @@ for skills. Its Tier 1 LLM checks still run only when requested.skillevaluator validate ./my-skill```- By default you get a compact pipeline view in the terminal — a per-check ticker per tier — and the run writes HTML and JSON report files unless you pass `-r` explicitly, which is then honored exactly (including `-r cli` for terminal-only output). Add `--verbose` for the full per-check detail stream instead of the compact view. The full pipeline needs the [provider and runtime setup](quickstart.mdx); use `--tiers 1` for static checks only. The exit code is `0` when every gate passes and `1` when one fails.+ By default you get a compact pipeline view in the terminal — a per-check ticker per tier — and the run writes HTML and JSON report files unless you pass `-r` explicitly, which is then honored exactly (including `-r cli` for terminal-only output). Add `--verbose` for the full per-check detail stream instead of the compact view. The full pipeline needs the [provider and runtime setup](/skills/skillevaluator/quickstart); use `--tiers 1` for static checks only. The exit code is `0` when every gate passes and `1` when one fails.### Run the complete offline gate@@ -108,7 +108,7 @@ for skills. Its Tier 1 LLM checks still run only when requested.Direct `tier1 PATH` and `tier1 validate PATH` print the terminal report andwrite JSON and HTML reports by default. Use `-r cli` for terminal-only output.-Reports for any of these runs are selected with `-r`/`--report` (`cli`, `json`, `html`, `markdown`, `sarif`) and written to `-o`/`--output-dir` (default `reports/`). File naming, `BENCHMARK.md`, and the JSON contract are covered in [Reports & Results](reports.mdx).+Reports for any of these runs are selected with `-r`/`--report` (`cli`, `json`, `html`, `markdown`, `sarif`) and written to `-o`/`--output-dir` (default `reports/`). File naming, `BENCHMARK.md`, and the JSON contract are covered in [Reports & Results](/skills/skillevaluator/reports).## Content types@@ -744,7 +744,7 @@ The command line rejects a malformed waiver file before the run starts, with exi## Scanner setup-Bandit installs with SkillEvaluator; install `skillevaluator[security]` for pip-audit, which the opt-in dependency audit uses. Install Semgrep and SkillSpector in separate tool environments, and install Gitleaks separately — it is a Go binary, not a pip package. See [Installation](installation.mdx) for commands and the extras table.+Bandit installs with SkillEvaluator; install `skillevaluator[security]` for pip-audit, which the opt-in dependency audit uses. Install Semgrep and SkillSpector in separate tool environments, and install Gitleaks separately — it is a Go binary, not a pip package. See [Installation](/skills/skillevaluator/installation) for commands and the extras table.Semgrep uses a policy packaged with the installed distribution, with metrics and version checks disabled; Tier 1 never fetches a registry policy at runtime. The other public Tier 1 configuration also ships inside the package under `skillevaluator/config/`: `pii_patterns.yaml`, `unicode_smuggle_patterns.yaml`, `license_config.yaml`, and `profiles/default.yaml`.@@ -815,7 +815,7 @@ Markdown and HTML reports show the same outcome as a `skillspector_baseline` linTier 1 checks always gate the exit code inside `validate`. Tier 2 gates when it runs unless `--no-block-on-dedup` is set. Tier 3 is advisory unless `--block-on-agent-eval` is set. The `quality` check requires the composite score to reach `--min-score` (default `70`) as well as no HIGH or CRITICAL quality findings under the active policy. Severity overrides cannot waive the minimum. Reports record the effective gate membership for every tier so saved evidence and process status agree.-`--fail-fast` stops after the first failed check. `-c`/`--continue-on-failure` overrides it, records the full pipeline, and keeps scanning a collection past a CRITICAL finding. Wiring these exit codes into a merge gate — including a complete GitHub Actions recipe — is covered in [Gate Your CI](ci-integration.mdx).+`--fail-fast` stops after the first failed check. `-c`/`--continue-on-failure` overrides it, records the full pipeline, and keeps scanning a collection past a CRITICAL finding. Wiring these exit codes into a merge gate — including a complete GitHub Actions recipe — is covered in [Gate Your CI](/skills/skillevaluator/ci-integration).## Troubleshooting@@ -906,7 +906,7 @@ Tier 1 checks always gate the exit code inside `validate`. Tier 2 gates when itEach stage gets the full watchdog, so a scan that stalls in both stages cantake up to twice the value before it is reported. See- [Environment Variables](environment-variables.mdx#validation-tier-1).+ [Environment Variables](/skills/skillevaluator/environment-variables#validation-tier-1).</Accordion><Accordion title="The SkillSpector LLM provider keeps failing">With `--llm`, SkillSpector v2.12.0 and later retries a failing provider call@@ -979,16 +979,16 @@ Tier 1 checks always gate the exit code inside `validate`. Tier 2 gates when it## Next steps<CardGroup cols={2}>- <Card title="Tier 2: Deduplication" icon="copy" href="tier2-deduplication.mdx">+ <Card title="Tier 2: Deduplication" icon="copy" href="/skills/skillevaluator/tier2-deduplication">Check whether your skill repeats itself or overlaps the rest of your collection.</Card>- <Card title="Gate Your CI" icon="shield" href="ci-integration.mdx">+ <Card title="Gate Your CI" icon="shield" href="/skills/skillevaluator/ci-integration">Turn the Tier 1 exit code into a merge gate with a copy-paste workflow.</Card>- <Card title="CLI Reference" icon="terminal" href="cli-reference.mdx">+ <Card title="CLI Reference" icon="terminal" href="/skills/skillevaluator/cli-reference">Every flag and default for validate and the standalone Tier 1 commands.</Card>- <Card title="Environment Variables" icon="book" href="environment-variables.mdx">+ <Card title="Environment Variables" icon="book" href="/skills/skillevaluator/environment-variables">Profile, scanner override, and identity variables in one table.</Card></CardGroup>diff --git a/docs/tier2-deduplication.mdx b/docs/tier2-deduplication.mdxindex e26fd3e1..760ce098 100644--- a/docs/tier2-deduplication.mdx+++ b/docs/tier2-deduplication.mdx@@ -8,8 +8,8 @@ Tier 2 finds redundant content — inside a single skill and across a skillcollection — using embedding similarity, with chat-LLM verification forintra-skill candidates. It needs a configured embeddings provider; intra-skillanalysis also needs a chat LLM. See-[Providers & Credentials](configuration.mdx) to set both up, including the-[fully local recipe](configuration.mdx#fully-local-setup) when skill content+[Providers & Credentials](/skills/skillevaluator/configuration) to set both up, including the+[fully local recipe](/skills/skillevaluator/configuration#fully-local-setup) when skill contentmust not leave your machine.Both checks read local files only.@@ -21,7 +21,7 @@ The [all-tier onboarding path](/skills/skillevaluator/advanced-ci)runs Tier 2 as an advisory check, independently from the Tier 1 image and pairedTier 3 evaluation. Configure chat and embedding provider credentials as CIsecrets. Tier 2 does not read `evals/`; committed native datasets are needed-only for [Tier 3](tier3-live-evaluation.mdx). Missing credentials or a central+only for [Tier 3](/skills/skillevaluator/tier3-live-evaluation). Missing credentials or a centraldataset mapping produce an actionable incomplete result before paid work.The aggregate artifact preserves native Tier 2 JSON and HTML, including@@ -55,7 +55,7 @@ Direct Tier 2 prints the terminal report and writes JSON and HTML reports toFor NVIDIA Build or OpenAI, the same provider and key configure both chat andembeddings; no separate embedding variables are needed. See-[Providers & Credentials](configuration.mdx) for other providers.+[Providers & Credentials](/skills/skillevaluator/configuration) for other providers.## Which check do you need@@ -242,7 +242,7 @@ again on a rerun, so Tier 2 records a MEDIUM `llm_refused` finding that namesthe overlapping sections and does not block. Review that overlap by hand, orrerun with another chat model (`--llm-model` or `SKILL_EVAL_LLM_MODEL`). To makea refusal block `validate`, raise `CONTENT_DEDUP.llm_refused` to `high` in a-[policy overlay](tier1-validation.mdx#validation-profiles-and-policies).+[policy overlay](/skills/skillevaluator/tier1-validation#validation-profiles-and-policies).The JSON report's `llm_analysis` names the configured `provider` and `model`.A server-side fallback model can answer a cluster in the configured model's@@ -482,7 +482,7 @@ All Tier 2 commands accept the standard report options (`-r cli,json,html,markdoand `-o`). HIGH and CRITICAL findings, along with blocking analysis errors,produce a non-zero exit code; MEDIUM and LOW findings are advisory. Exitbehavior is identical in every output mode, so these commands work as CI gates-— see [Gate Your CI](ci-integration.mdx).+— see [Gate Your CI](/skills/skillevaluator/ci-integration).Inside the combined `validate` pipeline, Tier 2 also gates by default. Pass`--no-block-on-dedup` to keep the scan and its findings in every report without@@ -559,7 +559,7 @@ scan.- Catalog files are read and written locally; SkillEvaluator does not uploadthem to a catalog service.-Use the [fully local provider recipe](configuration.mdx#fully-local-setup) when+Use the [fully local provider recipe](/skills/skillevaluator/configuration#fully-local-setup) whenskill content must not leave the machine, and review your provider's datahandling terms before sending confidential skill content to a hosted endpoint.@@ -604,13 +604,13 @@ handling terms before sending confidential skill content to a hosted endpoint.## Next steps<CardGroup cols={2}>- <Card title="Tier 3: Live Evaluation" icon="flask" href="tier3-live-evaluation.mdx">+ <Card title="Tier 3: Live Evaluation" icon="flask" href="/skills/skillevaluator/tier3-live-evaluation">Prove the skill changes agent behavior with a live A/B evaluation.</Card>- <Card title="Providers & Credentials" icon="book" href="configuration.mdx">+ <Card title="Providers & Credentials" icon="book" href="/skills/skillevaluator/configuration">Configure an embeddings provider — including a fully local setup.</Card>- <Card title="CLI Reference" icon="terminal" href="cli-reference.mdx">+ <Card title="CLI Reference" icon="terminal" href="/skills/skillevaluator/cli-reference">Every Tier 2 flag, default, and report option.</Card></CardGroup>diff --git a/docs/tier3-live-evaluation.mdx b/docs/tier3-live-evaluation.mdxindex cd484692..f03941bc 100644--- a/docs/tier3-live-evaluation.mdx+++ b/docs/tier3-live-evaluation.mdx@@ -2184,19 +2184,19 @@ machine-readable JSON contract, and how to judge whether a given lift is good.## Next steps<CardGroup cols={2}>- <Card title="Eval Datasets" icon="book" href="eval-datasets.mdx">+ <Card title="Eval Datasets" icon="book" href="/skills/skillevaluator/eval-datasets">The full evals/ contract — dataset format, generation, and config.yml runpolicy.</Card>- <Card title="Agents & Sandboxes" icon="shield" href="agents-and-sandboxes.mdx">+ <Card title="Agents & Sandboxes" icon="shield" href="/skills/skillevaluator/agents-and-sandboxes">Every supported agent and execution environment, plus readiness anddebugging tools.</Card>- <Card title="Custom Graders & Tasks" icon="flask" href="custom-graders.mdx">+ <Card title="Custom Graders & Tasks" icon="flask" href="/skills/skillevaluator/custom-graders">Bring your own grader or a complete Harbor task when the defaults can'texpress success.</Card>- <Card title="Reports & Results" icon="terminal" href="reports.mdx">+ <Card title="Reports & Results" icon="terminal" href="/skills/skillevaluator/reports">What every run writes to disk and how to read it — including the JSONcontract for CI.</Card>diff --git a/docs/trace-datasets.mdx b/docs/trace-datasets.mdxindex 0b94b4c6..796626fd 100644--- a/docs/trace-datasets.mdx+++ b/docs/trace-datasets.mdx@@ -118,4 +118,4 @@ is detected. Reviewers must still inspect all bundle files for sensitive data,missing replay inputs, and suitability for publication. Review decisions areexplicit local metadata, not authenticated signatures or proof of who reviewed.-See [Eval Datasets](eval-datasets.mdx) for the promoted dataset contract.+See [Eval Datasets](/skills/skillevaluator/eval-datasets) for the promoted dataset contract.