Skip to content

Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.

27.9 KB
Raw Blame
---
title: "Providers & Credentials"
description: "Configure LLM providers, embeddings, and agent credentials for SkillEvaluator — and see exactly which commands run with no key at all."
---

**For NVIDIA Build, getting started requires just two variables: the provider
and its API key.** Default chat and embedding models are already configured:

```bash title="One provider and key for all three tiers"
export SKILL_EVAL_LLM_PROVIDER=nv_build
export NVIDIA_API_KEY='nvapi-...'
```

The same key covers Tier 1 LLM checks, Tier 2 embeddings and chat, and supported
Tier 3 agents in Docker, Kata, or local mode. OpenAI likewise starts with
`SKILL_EVAL_LLM_PROVIDER=openai` and `OPENAI_API_KEY`, using Codex for Tier 3.
External scanners, agent runtimes, and sandboxes remain prerequisites; model
credentials do not install them.

```bash title="Run the tier you need"
skillevaluator tier1 ./my-skill
skillevaluator tier2 ./my-skill
skillevaluator tier3 ./my-skill
```

No model override is needed. Defaults are pinned in the installed release,
not automatically selected as the latest available model. Set
`SKILL_EVAL_LLM_MODEL` only when you want a different evaluator model; the
standard judge inherits it unless a judge override is set. Embeddings use their
own model, and Tier 3 agent selection is separate. With NVIDIA Build, all three
agents in SkillEvaluator 1.0.1 default to Nemotron 3.5 Lightning. OpenCode follows
`SKILL_EVAL_LLM_MODEL`, while Codex and Claude Code keep the Lightning default
unless you override their models.
PyPI 0.9.0 uses earlier defaults; see
[release differences](/skills/skillevaluator/installation-options#release-and-source-differences).

Anthropic and Bedrock need a separate embedding provider for Tier 2; custom
OpenAI-compatible endpoints require an explicit URL and key, with optional model overrides.
Those advanced setups are covered below.

For the exhaustive variable-by-variable inventory, see [Environment Variables](/skills/skillevaluator/environment-variables).

Tier 3 executes trials through
[Harbor](https://github.com/harbor-framework/harbor), the open-source agent
evaluation framework. This page separates the credentials SkillEvaluator and
the live agents need from the backend configuration Harbor itself reads.

## What needs a key

Tiers describe evaluation depth; model credentials are a separate concern.
Keyless checks do not need an LLM provider, although dependency audits can
contact package and vulnerability services.

| Runs without an LLM credential | Needs a provider key |
| --- | --- |
| `tier1 PATH --no-llm` and `validate PATH --tiers 1` (deterministic checks: schema, security scanners including secret detection, PII, license, code-integrity, unicode, quality, lint; direct Tier 1 also checks dependencies) | `rubric-eval` — LLM-as-judge scoring (chat LLM) |
| `quality-check`, `security-scan`, `pii-scan`, `lint-scripts` | `tier1 PATH` automatically enables LLM stages when configured; `validate --llm` / `security-scan --llm` — deeper LLM security analysis |
| `doctor`, `health-check` (offline readiness probes by default; missing credentials are reported as not ready) | `--llm-verify` on `validate`, `security-scan`, `pii-scan` — LLM false-positive suppression |
| | All Tier 2 commands — embeddings API; `context-optimization-check` and `dedup-scan` also use a chat LLM (`similarity-check` is embeddings-only). A local OpenAI-compatible server works too — see [Fully local setup](#fully-local-setup) |
| Tier 3 scaffolding and inspection: `create-eval-dataset PATH --no-llm --grounding skill`, `init-custom-grader`, `init-harbor-task`, `tier3 validate`, `view`, `harbor-view`, and `compare` | Tier 3 LLM dataset generation needs a provider key; `tier3 evaluate` needs a provider key plus a credential the selected live agent can use — with NVIDIA Build in `--env-mode docker` or `local`, one `NVIDIA_API_KEY` covers both roles |
| | `models` and `doctor --verify-models` — provider-catalog reachability and membership checks; public catalogs do not prove that an inference key works. `doctor --tier 2 --verify-models` sends one embedding and one chat request |

Standalone dataset generation defaults to substantive author guidance in
`evals/EVAL.md` or `--prompt`, then usable matching retained traces from a completed
skill-scoped run. Headings alone do not qualify. The table's explicit
`--grounding skill` opts into skill-only templates; see
[Eval Datasets](/skills/skillevaluator/eval-datasets#generate-a-dataset).

`validate PATH` runs all three tiers for skills and needs the provider and
Tier 3 runtime setup. Use `--tiers 1` for keyless static checks. `--no-dedup`
alone skips Tier 2 but still runs Tier 3.

Before Tier 3 prepares images or stages Harbor tasks, `tier3 evaluate` also
makes a bounded credential-bearing catalog request for each distinct agent route
and for the standard-grading route when that grader's effective model is known.
A definitive native-provider credential rejection or deterministic Bedrock
credential/configuration error fails immediately. Catalog permission and
membership are not authoritative for OpenAI Responses access, and NVIDIA Build
or a compatible gateway may expose a catalog without authenticating inference,
so those checks are reported as degraded and evaluation continues. When enabled
with `--agent-runtime-preflight` or `harbor.agent_runtime_preflight: true`, the
extra agent-only execution proves agent routes only: when a
degraded credential route is used solely by the standard grader, inference on
that route remains unproven until grading begins. The same applies when native
Harbor must resolve at runtime whether a task or step verifier declaration
supersedes the configured judge fallback. Redacted per-route outcomes are retained under
`run_config.credential_validation` in `result.json`, including when an enabled
agent runtime preflight fails. The extra execution is disabled by default.

## Choose an LLM provider

Set `SKILL_EVAL_LLM_PROVIDER` to one of `openai`, `anthropic`, `nv_build`, `bedrock`, or `openai-compatible`. The selected provider determines which credential SkillEvaluator reads. Each ships a default evaluator model that `SKILL_EVAL_LLM_MODEL` overrides.

<Tabs>
  <Tab title="OpenAI" language="openai">
    ```bash title="OpenAI setup"
    export SKILL_EVAL_LLM_PROVIDER=openai
    export OPENAI_API_KEY='sk-...'
    ```

    Default model: `gpt-6.1-sol` (OpenAI GPT-6.1 Sol frontier). For a lower-cost override set `SKILL_EVAL_LLM_MODEL=gpt-5.4-mini`. Endpoint `https://api.openai.com/v1`. Override the endpoint with `SKILL_EVAL_LLM_BASE_URL` (takes precedence) or `OPENAI_BASE_URL` — but a base URL that is a non-canonical alias of the native OpenAI endpoint is rejected; use `https://api.openai.com/v1` verbatim. On the canonical endpoint, GPT-5 and GPT-6 models automatically send `max_completion_tokens` instead of `max_tokens`. Tier 3's OpenCode agent uses `openai/gpt-5.6-sol` unless `SKILL_EVAL_LLM_MODEL` is set, because OpenCode calls tools through Chat Completions and GPT-6.1 Sol supports tool calls only on the Responses API.
  </Tab>
  <Tab title="Anthropic" language="anthropic">
    ```bash title="Anthropic setup"
    export SKILL_EVAL_LLM_PROVIDER=anthropic
    export ANTHROPIC_API_KEY='sk-ant-...'
    ```

    Default model: `claude-opus-5-5`, using the Anthropic SDK's default endpoint. Override it with `SKILL_EVAL_LLM_BASE_URL` (highest precedence) or `ANTHROPIC_BASE_URL`; when neither is set, SkillEvaluator uses the Anthropic default.

    For Anthropic, either base-URL variable must be an absolute HTTP(S) API root. Supply an unversioned root such as `https://gateway.example.com` or a legacy URL whose final segment is `/v1`, such as `https://gateway.example.com/team/v1`. SkillEvaluator normalizes the latter to its API root so the Anthropic SDK, the Tier 3 verifier, and a matching Claude Code agent each form request paths with exactly one `/v1/messages` suffix. An independently credentialed Claude Code route applies the same normalization and validation to `ANTHROPIC_BASE_URL` when the evaluator uses another provider. Routing through path prefixes, ports, IP addresses, internationalized domain names, and ordinary safe percent escapes is preserved.

    Validation fails before Harbor starts if the URL contains a full `/v1/messages` endpoint, embedded credentials, a query or fragment, whitespace or control characters, a backslash, an invalid authority, interior empty or dot path segments, or ambiguous multi-layer endpoint encoding. This root convention is specific to Anthropic; OpenAI-compatible endpoints commonly retain a terminal `/v1` unchanged.
  </Tab>
  <Tab title="NVIDIA Build" language="nv_build">
    ```bash title="NVIDIA Build setup"
    export SKILL_EVAL_LLM_PROVIDER=nv_build
    export NVIDIA_API_KEY='nvapi-...'
    ```

    Default model: `nvidia/nemotron-3.5-lightning-30b-a3b`, shared by evaluator chat, Tier 3's default OpenCode agent, and the standard judge. The endpoint is fixed to `https://integrate.api.nvidia.com/v1` — `SKILL_EVAL_LLM_BASE_URL` and `SKILL_EVAL_EMBEDDING_BASE_URL` do not override it. To point at a custom or self-hosted endpoint, use the `openai-compatible` provider instead.

    The previous default, `nvidia/nemotron-3-nano-30b-a3b`, has a deprecated public endpoint. If you explicitly pinned that model in `SKILL_EVAL_LLM_MODEL`, remove the override to use the current default. Explicit model selections are preserved; SkillEvaluator does not silently switch models after a failed request.

    <Tip>
    The quickest start is a free NVIDIA API Catalog key from [build.nvidia.com](https://build.nvidia.com/), exported as `NVIDIA_API_KEY` — it covers LLM judging and Tier 2 embeddings with one credential.
    </Tip>
  </Tab>
  <Tab title="Bedrock" language="bedrock">
    ```bash title="Amazon Bedrock setup"
    export SKILL_EVAL_LLM_PROVIDER=bedrock
    export AWS_REGION=us-west-2   # default when unset
    # Authenticate with the standard AWS credential chain
    # (environment variables, shared profile, or an assumed role).
    ```

    Default model: `us.anthropic.claude-opus-5-5` on the AWS Bedrock Runtime (US geo inference profile for Claude Opus 5.5).
  </Tab>
  <Tab title="OpenAI-compatible" language="openai-compatible">
    ```bash title="OpenAI-compatible setup"
    export SKILL_EVAL_LLM_PROVIDER=openai-compatible
    export SKILL_EVAL_LLM_BASE_URL='http://your-server:8000/v1'
    export SKILL_EVAL_LLM_API_KEY='your-key'
    ```

    These three variables select the gateway. Model overrides are optional when the gateway serves the defaults below; no endpoint or credential is built into SkillEvaluator.

    | Role | Default model | Override |
    | --- | --- | --- |
    | Evaluator and standard judge | `nvidia/nvidia/nemotron-3-super-120b-long-ctx` | `SKILL_EVAL_LLM_MODEL`; judge-only: `LLM_JUDGE_MODEL` then `SKILL_EVAL_JUDGE_MODEL` |
    | Tier 2 embeddings | `nvidia/nvidia/nemotron-3-embed-1b` | `SKILL_EVAL_EMBEDDING_MODEL` |
    | Codex agent | `openai/openai/gpt-6.1-sol` | `--agent-model codex=MODEL` |
    | Claude Code agent | `aws/anthropic/bedrock-claude-opus-5-5` | `--agent-model claude-code=MODEL` |
    | OpenCode agent | `nvidia/nvidia/nemotron-3-super-120b-long-ctx` | `--agent-model opencode=openai/MODEL` |

    Gateways can use different catalog IDs. Supply those exact IDs through the overrides; SkillEvaluator never substitutes a different model after a request fails. OpenCode's `openai/` prefix selects its adapter and is removed before sending the catalog ID. `SKILL_EVAL_LLM_MODEL` controls evaluation and judging; it does not replace these separate gateway agent defaults. Agent selection uses `--agent-model` / `--model`, then `harbor.agents.AGENT.model`, then the harness default, and records the model source in the run report.

    For Tier 3 in Docker, OpenCode requires Chat Completions, Codex requires Responses, and Claude Code requires Anthropic Messages, each with tool-call support. Selecting a shared URL does not add those APIs to a server. See the [Claude gateway requirements](https://code.claude.com/docs/en/llm-gateway).

    Claude Code inherits the shared key and URL, stripping a terminal `/v1` from its API root while retaining any path prefix. An explicit `ANTHROPIC_BASE_URL` overrides that root. With the shared key, Claude Code runs with `CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1`, so it does not send pre-release request fields that a gateway forwarding to another provider rejects. An explicit `ANTHROPIC_API_KEY` selects an independent Claude route: without `ANTHROPIC_BASE_URL`, it keeps the native Anthropic endpoint and defaults to `claude-opus-5-5`. CLI and skill-config model overrides still take precedence. Existing native-provider routes are unchanged.
  </Tab>
</Tabs>

### Auto-selection

When `SKILL_EVAL_LLM_PROVIDER` is unset and exactly one of these credentials is present, SkillEvaluator selects the matching provider:

- `NVIDIA_API_KEY` → `nv_build`
- `OPENAI_API_KEY` → `openai`
- `ANTHROPIC_API_KEY` → `anthropic`

If none of those variables is set, LLM-backed commands fail with a configuration error. If more than one is exported, resolution also fails — `SKILL_EVAL_LLM_PROVIDER` is required when multiple public provider credentials are configured; there is no silent preference order.

## Embeddings

Tier 2 uses an OpenAI-compatible embeddings API. `SKILL_EVAL_EMBEDDING_PROVIDER` selects it; when unset, it falls back to your `SKILL_EVAL_LLM_PROVIDER` (or the same auto-selection order as above).

| Provider (`SKILL_EVAL_EMBEDDING_PROVIDER`) | Credential | Default embedding model |
| --- | --- | --- |
| `openai` | `OPENAI_API_KEY` | `text-embedding-3-small` |
| `nv_build` | `NVIDIA_API_KEY` | `nvidia/nemotron-3-embed-1b` |
| `openai-compatible` | `SKILL_EVAL_EMBEDDING_API_KEY` (falls back to `SKILL_EVAL_LLM_API_KEY`) | `nvidia/nvidia/nemotron-3-embed-1b` |

Use `SKILL_EVAL_EMBEDDING_MODEL` and `SKILL_EVAL_EMBEDDING_BASE_URL` to override defaults; for `openai-compatible`, the base URL falls back to `SKILL_EVAL_LLM_BASE_URL`. The `nv_build` endpoint is fixed to `https://integrate.api.nvidia.com/v1` — only its model can be overridden.

NVIDIA Build uses the same `NVIDIA_API_KEY` for chat and embeddings; no extra key is needed. Deduplication sends document inputs with `input_type=passage`, as required by the [NVIDIA embedding API](https://docs.api.nvidia.com/nim/reference/nvidia-nemotron-3-embed-1b-infer).

The previous NVIDIA default, `nvidia/nv-embed-v1`, is retired. Remove any explicit override to that model to use the current default. Rebuild comparison catalogs and embedding caches when changing models: vectors from different models cannot be compared, and similarity scores may change. A provider failure is reported as **INCOMPLETE**, not a duplicate-content finding.

<Note>
Anthropic and Bedrock do not provide embeddings, and SkillEvaluator rejects them as embedding providers. When one of those is your LLM provider, configure an embedding-capable provider separately for Tier 2:

```bash title="Separate embedding provider"
export SKILL_EVAL_EMBEDDING_PROVIDER=openai
export OPENAI_API_KEY='sk-...'
```
</Note>

## Fully local setup

The `openai-compatible` provider accepts any local OpenAI-compatible server (Ollama, vLLM, llama.cpp, NVIDIA NIM), so Tier 2 can run with no external calls at all. Example recipe using Ollama in Docker:

<Steps>
  ### Start a local model server

  ```bash title="Ollama in Docker"
  docker run -d --name ollama -p 11434:11434 ollama/ollama
  docker exec ollama ollama pull nomic-embed-text   # embeddings
  docker exec ollama ollama pull qwen2.5:0.5b       # chat LLM for dedup analysis
  ```

  ### Point SkillEvaluator at it

  ```bash title="Local provider configuration"
  export SKILL_EVAL_EMBEDDING_PROVIDER=openai-compatible
  export SKILL_EVAL_EMBEDDING_BASE_URL=http://localhost:11434/v1
  export SKILL_EVAL_EMBEDDING_MODEL=nomic-embed-text
  export SKILL_EVAL_EMBEDDING_API_KEY=local-no-key  # must be set; local servers ignore the value

  # context-optimization-check and dedup-scan also use a chat LLM:
  export SKILL_EVAL_LLM_PROVIDER=openai-compatible
  export SKILL_EVAL_LLM_BASE_URL=http://localhost:11434/v1
  export SKILL_EVAL_LLM_MODEL=qwen2.5:0.5b
  export SKILL_EVAL_LLM_API_KEY=local-no-key
  ```

  ### Run the Tier 2 checks

  ```bash title="Fully local Tier 2"
  skillevaluator similarity-check ./skills
  skillevaluator context-optimization-check ./my-skill
  ```
</Steps>

`similarity-check` compares skills directly or through a local catalog and needs only the embedding variables. `context-optimization-check` (also available as its `dedup-scan` alias) additionally uses the chat model. Analysis quality tracks that model — the tiny model above proves the plumbing; pick a stronger local model for verdicts you intend to act on.

## Tier 3 credentials: two roles

Live evaluation has two credential roles. They can share a key when the selected provider supports both:

| Role | Who uses it | How you configure it |
| --- | --- | --- |
| Evaluator key | SkillEvaluator itself — dataset generation and verifier-side judging | The provider setup above (`SKILL_EVAL_LLM_PROVIDER` + its key) |
| Agent credential | The live agent under test (Codex, Claude Code, ...) inside its task environment | Your host environment only — export the agent's native variables (for example `OPENAI_API_KEY` and `OPENAI_BASE_URL` for Codex) before the run |

Both roles resolve from the operator's environment; a skill's `evals/config.yml` can never supply, rename, or redirect a credential. Export them in the shell or CI job that runs SkillEvaluator: Harbor starts in an empty, evaluator-owned working directory with `.env` loading disabled, so a `.env` or `.env.local` file near your project is not read. What each agent needs depends on the provider and `--env-mode`:

- **NVIDIA Build, one key** (`--env-mode docker`, `kata`, or `local`): a single `NVIDIA_API_KEY` powers the evaluator and all three supported agents — `opencode` calls NVIDIA Build directly, while `claude-code` and `codex` route through SkillEvaluator's compatibility bridge (in-container in Docker and Kata modes; an authenticated in-process host bridge in `local` mode). Agent models must be full NVIDIA Build catalog IDs in `publisher/model` form.
- **NVIDIA Build on Harbor cloud modes** (`daytona`, `e2b`, ...): the bridge is not available. `claude-code` needs its own native `ANTHROPIC_API_KEY`, `codex` needs a full OpenAI Responses API credential (`OPENAI_API_KEY` plus `OPENAI_BASE_URL`), and each needs an explicit model via `--agent-model` or `harbor.agents.<agent>.model`.
- **Matching native provider**: when the evaluator provider is the agent's own (for example `anthropic` with `claude-code`, or `openai` with `codex`), the same key serves both roles automatically.
- **OpenAI-compatible gateway**: Codex, OpenCode, and Claude Code inherit the shared URL and key. The gateway must support the selected harness's API and model. Explicit Claude credentials and endpoint settings take precedence as described above.

When `--agents` is omitted, SkillEvaluator chooses that native route:
NVIDIA Build uses OpenCode, OpenAI uses Codex, and Anthropic uses Claude Code.
Passing `--agents` always replaces the automatic selection.

### Judge failures are not zero scores

Standard Tier 3 grading treats a required judge transport, authentication, timeout, HTTP, or exhausted response-parsing error as an evaluation infrastructure failure. A condition containing a failed or incomplete default trial publishes none of its scores, dimensions, pass@k, or quality findings, and no cross-condition lift is published. Redacted diagnostics and coverage information remain available. A genuine model score of `0.0` is still a valid, published score.

In `default_plus_custom` grading, a custom score cannot rescue failed standard judging. `custom_only` grading is unchanged: it does not invoke the standard judge and continues to publish a valid custom result.

The agent's model lives in the skill's `evals/config.yml`:

```yaml title="evals/config.yml"
schema_version: 1
harbor:
  agents:
    codex:
      model: gpt-6.1-sol
```

You can also supply that model at invocation time with `--agent-model codex=MODEL`. See [Tier 3: Live Agent Evaluation](/skills/skillevaluator/tier3-live-evaluation) for the full run workflow.

In Docker mode, credential values are never rendered on the `docker compose exec` command line — the bridge receives them through a mode-0600 file inside the container that is consumed and removed before the agent starts.

### List the models your key can access

Before pinning `--agent-model` or `SKILL_EVAL_LLM_MODEL`, check what the selected provider actually serves:

```bash title="List provider models"
skillevaluator models --limit 20
skillevaluator models --json
```

`skillevaluator models` prints a filtered, bounded view of the provider catalog (`--limit` accepts 1–100, default 10; `--json` emits machine-readable output). It uses the same provider and credential resolution as evaluation, so it checks catalog routing and model discovery where that surface is supported. NVIDIA Build and some compatible gateways can expose a public catalog, so a listed model is not proof that the inference key works or that a given agent harness supports it. Tier 3's optional real per-agent runtime preflight is the stronger check for agent routes; enable it with `--agent-runtime-preflight` when that extra billable execution is appropriate. A separate degraded standard-grader route remains unproven until grading begins. Bedrock exposes no HTTP model catalog, so `models` errors on that provider; use `doctor --verify-models` there instead.

### `runtime_env` is for task values, not credentials

`runtime_env` in `evals/config.yml` remains valid for non-credential task values — plain names the task environment needs. Credential names are rejected outright:

- Any operator-owned credential name (`OPENAI_API_KEY`, `OPENAI_BASE_URL`, `ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`, `NVIDIA_API_KEY`, `CLAUDE_CODE_USE_BEDROCK`) fails the run with `harbor.runtime_env contains operator-owned credential name(s)`.
- A `${NVIDIA_API_KEY}`-style reference to an operator-owned credential inside any value is rejected too, so a credential cannot be aliased under a different name.

### Blocked `runtime_env` names

Names that control the launcher, credentials, or dynamic runtime are rejected:

| Blocked group | Examples |
| --- | --- |
| Operator-owned agent credentials | `OPENAI_API_KEY`, `OPENAI_BASE_URL`, `ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`, `NVIDIA_API_KEY`, `CLAUDE_CODE_USE_BEDROCK` — as names or as `${...}` references in values |
| Host path and loader control | `PATH`, `PATHEXT`, `LD_*`, `DYLD_*`, `PYTHON*` |
| Container and backend selection | `DOCKER_*`, `COMPOSE_*`, `HARBOR_*`, plus each Harbor backend's own credential variables |
| AWS credential chain | `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`, `AWS_PROFILE`, `AWS_ROLE_ARN`, and the rest of the `AWS_*` credential-chain variables |
| Network and TLS control | `HTTP_PROXY`, `HTTPS_PROXY`, `NO_PROXY`, `ALL_PROXY`, `SSL_CERT_FILE`, `SSL_CERT_DIR`, `REQUESTS_CA_BUNDLE`, `SSLKEYLOGFILE`, `SSH_AUTH_SOCK` |
| Tracing and tooling | `OTEL_*`, `SKILL_EVAL_*`, `SKILLEVALUATOR_*`, `GIT_*`, `NODE_*`, `PIP_*`, `UV_*` |

Configure the selected Harbor backend in the host environment instead.
Non-secret constructor kwargs are only for Harbor-native non-Docker backends.
They are operator-only: use repeatable `--environment-kwarg KEY=VALUE` / `--ek
KEY=VALUE` flags on `tier3 evaluate`, `doctor`, and `health-check`.
Skill-owned `evals/config.yml` cannot set them. `docker` and `local` reject any
environment kwargs, and `kata` accepts only `kata_runtime` and `kata_dns`. Harbor runtime-policy fields for resource overrides,
mounts, networks, extra Compose definitions, SkillEvaluator's own lifecycle
fields (`delete`, `force_build`, `keep_containers`), Harbor's `stream` and
`enable_environment_dir_upload` controls, and Kubernetes pod security are
reserved. Other backend-specific timeout and retention knobs remain
operator-owned. Never pass secrets there; credentials remain host environment
variables.

## SkillSpector bridge

The `--llm` security analysis runs [SkillSpector](https://github.com/NVIDIA/SkillSpector), which has its own provider environment (`SKILLSPECTOR_PROVIDER` plus per-provider credential variables). SkillEvaluator bridges your configured provider automatically for that invocation, so the one key above is all you need. For `nv_build`, the bridge selects SkillSpector's `nv_build` provider, so its Build model registry sets output-token limits, and sends the key through SkillSpector's OpenAI-compatible provider path (`OPENAI_API_KEY`, with `OPENAI_BASE_URL` pointed at the public Build endpoint). The same `NVIDIA_API_KEY` covers this too; it does not create a second NVIDIA credential name.

The subprocess environment is isolated. Only the selected provider settings and basic process environment variables (`PATH`, `HOME`, TLS bundles, temp dirs) pass through — other ambient credentials in your shell are never forwarded. Your SkillSpector model settings (`SKILLSPECTOR_MODEL_<SLOT>`, `SKILLSPECTOR_MODEL_REGISTRY`, `SKILLSPECTOR_REASONING_EFFORT`) pass through too; `SKILLSPECTOR_TEMPERATURE` does not.

SkillSpector sends its own structured-output requests, so it has its own default model per provider instead of following the chat default. On Anthropic and Bedrock the default is Claude Opus 5.5, the same as the chat default; SkillSpector 2.13.4 returns structured output for Claude 5.5 models. An explicit `SKILL_EVAL_LLM_MODEL` still reaches SkillSpector, and `SKILLSPECTOR_MODEL` overrides both. See [SkillSpector bridge](/skills/skillevaluator/environment-variables#skillspector-bridge) for the defaults.

Setting `SKILLSPECTOR_PROVIDER` yourself overrides the bridge, but it must name a supported public provider (`openai`, `anthropic`, or `bedrock` — or `nv_build` to force the Build mapping); anything else fails closed and the subprocess receives no credential.

## Troubleshooting

<AccordionGroup>
  <Accordion title="&quot;No provider is configured (SKILL_EVAL_LLM_PROVIDER unset and no credential found)&quot;">
    No provider is selected and no known key is present. Export one of `NVIDIA_API_KEY`, `OPENAI_API_KEY`, or `ANTHROPIC_API_KEY`, or set `SKILL_EVAL_LLM_PROVIDER` explicitly alongside its credential.
  </Accordion>
  <Accordion title="&quot;SKILL_EVAL_LLM_PROVIDER is required when multiple public provider credentials are configured&quot;">
    More than one of `NVIDIA_API_KEY`, `OPENAI_API_KEY`, and `ANTHROPIC_API_KEY` is exported, so auto-selection refuses to guess. Set `SKILL_EVAL_LLM_PROVIDER` to the provider you want (single-key auto-detection is unaffected). Relatedly, an exported-but-empty `SKILL_EVAL_LLM_MODEL` is also an error — unset it to use the provider default, or set a model name.
  </Accordion>
  <Accordion title="&quot;SKILL_EVAL_EMBEDDING_PROVIDER is required because anthropic does not provide embeddings&quot;">
    Your LLM provider is `anthropic` or `bedrock`, and neither offers an embeddings API. Set `SKILL_EVAL_EMBEDDING_PROVIDER` to `openai`, `nv_build`, or `openai-compatible` with the matching credential — see [Embeddings](#embeddings).
  </Accordion>
  <Accordion title="A gateway rejects the default model or API">
    Check your gateway's catalog and API support. Override `SKILL_EVAL_LLM_MODEL`, `SKILL_EVAL_EMBEDDING_MODEL`, or `--agent-model AGENT=MODEL` with its exact model IDs. OpenCode uses Chat Completions, Codex uses Responses, and Claude Code uses Anthropic Messages; a server exposing only one API cannot run all three harnesses. Older releases requiring an explicit model must be upgraded to use these defaults. Local servers still require the key variable even if they ignore its value.
  </Accordion>
</AccordionGroup>

## Next steps

<CardGroup cols={2}>
  <Card title="Tier 2: Deduplication" icon="copy" href="/skills/skillevaluator/tier2-deduplication">
    Put your embedding provider to work — find overlap within and across skills.
  </Card>
  <Card title="Tier 3: Live Agent Evaluation" icon="flask" href="/skills/skillevaluator/tier3-live-evaluation">
    Run a live agent evaluation with the two-credential setup above.
  </Card>
</CardGroup>