Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.
Python agent-evaluationagent-pluginsagent-securityagent-skillsagentic-aibenchmarkci-templatesclaude-codecodexevaluateevaluationharnesspluginplugin-evaluationsecurity-scannerskill-evalskill-evaluationskill-evaluatorskill-gap-analysisskills
feat(harness): add the harness-check command and compatibility panel (#521)
Christopher Kevin976f91f508cb
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 695ff8ae..f40d2a02 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -6,6 +6,13 @@ All notable changes to SkillEvaluator are documented in this file.### Added+- New `skillevaluator harness-check <skill>` command and a "Harness compatibility"+ panel after the Tier 1 results of `validate`: one line per harness with its most+ serious finding and an outcome label (`not found`, `rejected`, `listing+ shortened`, ...), then `+N more` and the note count; `-v` shows every finding+ with its fix. The checks line shows `! harness`, Tier 1 gets a `harness` summary+ line, and the result box names the harnesses that won't load the skill.+ `harness-check` always exits 0; `--json PATH` writes the block.- New advisory Tier 1 `harness` check: for each agent harness (`claude-code`,`codex`, `copilot-cli`, `hermes`, `openclaw`, `opencode`) it reports whether theskill `loads`, is `degraded` or is `not_loaded`, from versioned profiles shippeddiff --git a/docs/tier1-validation.mdx b/docs/tier1-validation.mdxindex 7db9647c..15fc909b 100644--- a/docs/tier1-validation.mdx+++ b/docs/tier1-validation.mdx@@ -270,6 +270,8 @@ For `validate`, the default set is `schema`, `version`, `security`, `pii`, `lice`validate` runs it by default for all six and lists its findings in the HTML, JSON and SARIF reports; JSON reports also get a `harness_compatibility` block. Use `--checks harness` to run it alone and `--harness codex,claude-code` to narrow the harnesses.The profiles ship with the package and come from repeated runs of each harness in which the request sent to the model was captured. The check doesn't change pass/fail: its findings are MEDIUM (`not_loaded`), LOW (`degraded`) or INFO (`advisory`), so it never fails a run unless a policy raises those severities. Install-location rules apply only when the skill sits under a harness root such as `.claude/skills` or `.agents/skills`. Use `--harness-profiles DIR` to try newer profile YAMLs without a release.++ `validate` also shows a "Harness compatibility" panel after Tier 1, `! harness` in the checks line when a harness drops or degrades the skill, a `harness` line in the Tier 1 summary (kept out of the advisories count) and a note in the result box. `skillevaluator harness-check <skill>` runs only this check and shows the same panel, updated as each harness is checked. It accepts a folder of skills, `--harness`, `-v` and `--json PATH`, always exits 0, and points to `validate --tiers 1` when Tier 1 schema also fails. By default the panel is compact: one line per harness with its most serious finding (not loaded, then degraded, then note; ties go to the rule listed first in the profile) labelled with its outcome (`not found`, `rejected`, `listing shortened`, `delivery truncated` and so on), then `+N more` and the number of notes. `-v` (also on `validate`) lists every finding with its fix and each harness's verified version.</Accordion><Accordion title="security — vulnerability scanning">diff --git a/src/skillevaluator/cli.py b/src/skillevaluator/cli.pyindex 86c12885..825164fe 100644--- a/src/skillevaluator/cli.py+++ b/src/skillevaluator/cli.py@@ -28,7 +28,7 @@ from skillevaluator.constants import (SIMILARITY_MAX_ENTRIES,)from skillevaluator.doctor import DEFAULT_PROBE_TIMEOUT_SEC, MAX_PROBE_TIMEOUT_SEC-from skillevaluator.harness.report import harness_block, write_json_block+from skillevaluator.harness.report import harness_block, harness_note, harness_panel, write_json_blockfrom skillevaluator.logging_config import setup_loggingfrom skillevaluator.models.result import ValidationResultfrom skillevaluator.quality_config import QualityConfigError, load_quality_config@@ -255,7 +255,7 @@ _TOP_LEVEL_COMMAND_HELP_GROUPS = (("Repository CI", ("ci",)),("Tier 1 · Static and security",- ("quality-check", "rubric-eval", "security-scan", "pii-scan", "lint-scripts"),+ ("quality-check", "rubric-eval", "security-scan", "pii-scan", "lint-scripts", "harness-check"),),("Tier 2 · Deduplication",@@ -2776,6 +2776,8 @@ def validate(apply_policy(results, policy)tier1_ok, tier1_rows = summarize_tier1(results, lineup=check_lineup)harness_rows = harness_block(results)+ if harness_rows:+ view.set_harness(harness_panel(harness_rows, width=view.width, verbose=verbose), harness_note(harness_rows))view.tier_done(0, failed=not tier1_ok, rows=tier1_rows)tier1_gate_results = list(results)tier2_gate_results: list[ValidationResult] = []@@ -3067,6 +3069,8 @@ def validate(agent_eval=agent_eval,presentation_context=presentation_context,)+ if harness_rows and not quiet:+ console.print(harness_panel(harness_rows, verbose=True))if gate_failed:raise click.ClickException("validation failed")@@ -3080,6 +3084,58 @@ validate.help_group_descriptions = {}[email protected]("harness-check")+@_target_argument+ "--harness",+ default="all",+ show_default=True,+ help="Comma-separated harnesses to check, e.g. codex,claude-code.",+)+ "--harness-profiles",+ type=click.Path(path_type=Path, exists=True, file_okay=False),+ default=None,+ help="Directory of harness profile YAMLs to use instead of the packaged ones.",+)[email protected]("-v", "--verbose", is_flag=True, help="List every harness with its verified version and date.")+ "--json",+ "json_path",+ type=click.Path(path_type=Path, dir_okay=False),+ default=None,+ help="Also write the harness_compatibility block to this JSON file.",+)+def harness_check(+ target_path: Path,+ harness: str,+ harness_profiles: Path | None,+ verbose: bool,+ json_path: Path | None,+) -> None:+ """Show whether each agent harness loads, degrades or drops a skill (advisory; exits 0)."""+ from skillevaluator.harness.report import run_harness_check+ from skillevaluator.reporting.console_ui import make_view_console++ try:+ block = run_harness_check(target_path, harness, harness_profiles, console=make_view_console(), verbose=verbose)+ except ValueError as exc:+ raise click.BadParameter(str(exc), param_hint="--harness") from exc+ if not block:+ raise click.ClickException(f"No skills found in {target_path}")+ from skillevaluator.validators.schema import SchemaValidator++ for skill in block:+ schema = SchemaValidator().validate(Path(skill))+ if not schema.passed and schema.errors:+ click.echo(+ f"Tier 1 schema also fails for {Path(skill).name}: {schema.errors[0].split(' in /', 1)[0]}. "+ "Run `skillevaluator validate --tiers 1` for details."+ )+ if json_path is not None:+ json_path.write_text(json.dumps({"harness_compatibility": block}, indent=2), encoding="utf-8")++@cli.command("quality-check")@_target_argument@click.option("--min-score", type=int, default=QUALITY_DEFAULT_MIN_SCORE, show_default=True)diff --git a/src/skillevaluator/harness/report.py b/src/skillevaluator/harness/report.pyindex df67d1e9..d439d451 100644--- a/src/skillevaluator/harness/report.py+++ b/src/skillevaluator/harness/report.py@@ -1,19 +1,44 @@# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.# SPDX-License-Identifier: Apache-2.0-"""Collect the advisory harness-compatibility rows and write them to the JSON report."""+"""Render the advisory harness-compatibility panel and JSON block."""from __future__ import annotationsimport json+import timefrom collections.abc import Iterablefrom pathlib import Pathfrom typing import TYPE_CHECKING, Any-from skillevaluator.validators.harness import HARNESS_METADATA_KEY+from rich import box+from rich.console import Console, Group+from rich.live import Live+from rich.panel import Panel+from rich.table import Table+from rich.text import Text++from skillevaluator.harness.overlay import evaluate_profile, facts_for, load_profiles+from skillevaluator.reporting.console_ui import AMBER, BRIGHT, FAINT, GREEN, MUTED, RED, SPIN, TEXT, WIDTH+from skillevaluator.validators.harness import HARNESS_METADATA_KEY, HarnessValidatorif TYPE_CHECKING:from skillevaluator.models.result import ValidationResult+FOOTER = "Doesn't change pass/fail."+VERBOSE_HINT = " Details and fixes: -v"+_STATUS = {+ "loads": ("✓", GREEN, "loads", TEXT),+ "degraded": ("!", AMBER, "degraded", f"bold {AMBER}"),+ "not_loaded": ("✗", RED, "NOT LOADED", f"bold {RED}"),+ "pending": ("○", FAINT, "", FAINT),+}+_EFFECT = {+ "not_loaded": ("✗", RED, "NOT LOADED"),+ "degraded": ("!", AMBER, "degraded"),+ "advisory": ("·", MUTED, "note"),+}+_RANK = {"advisory": 0, "degraded": 1, "not_loaded": 2}+Block = dict[str, list[dict[str, Any]]]@@ -25,9 +50,186 @@ def harness_block(results: Iterable[ValidationResult]) -> Block:return block+def harness_note(block: Block) -> str | None:+ """One-line verdict note naming the harnesses that drop or degrade the skill(s)."""+ parts = []+ for status, verb in (("not_loaded", "won't load in"), ("degraded", "loads degraded in")):+ skills = [skill for skill, rows in block.items() if any(row["status"] == status for row in rows)]+ names = sorted({row["harness"] for rows in block.values() for row in rows if row["status"] == status})+ if names:+ subject = "skill" if len(skills) == 1 else f"{len(skills)} skills"+ parts.append(f"{subject} {verb} {', '.join(names)}")+ return " · ".join(parts) or None+++def _glyph(row: dict[str, Any]) -> tuple[str, str]:+ if row["status"] == "checking":+ return SPIN[int(time.time() * 8) % len(SPIN)], f"bold {BRIGHT}"+ glyph, color, _, _ = _STATUS[row["status"]]+ return glyph, f"bold {color}"+++def _summary_line(rows: list[dict[str, Any]], prefix: str = "") -> Text:+ line = Text(prefix, style=f"bold {TEXT}")+ for index, row in enumerate(rows):+ glyph, style = _glyph(row)+ line.append(" " if index else "", style="")+ line.append(f"{glyph}\u00a0", style=style)+ line.append(row["harness"], style=TEXT if row["status"] not in ("pending", "checking") else MUTED)+ return line+++def _plural(count: int, word: str) -> str:+ return f"{count} {word}{'s' if count != 1 else ''}"+++def _compact_grid(rows: list[dict[str, Any]]) -> Table | None:+ """One line per flagged harness: its most serious finding, then how many more."""+ grid = Table.grid(padding=(0, 2), expand=True)+ grid.add_column(no_wrap=True)+ grid.add_column(ratio=1, style=TEXT)+ grid.add_column(no_wrap=True, style=MUTED, justify="right")+ for row in rows:+ findings = row.get("findings") or []+ if row["status"] in ("pending", "checking") or not findings:+ continue+ worst = max(findings, key=lambda finding: _RANK[finding["effect"]])+ problems = sum(finding["effect"] != "advisory" for finding in findings)+ notes = len(findings) - problems+ glyph, color, label, _ = _STATUS[row["status"]]+ more = [f"+{problems - 1} more"] if problems > 1 else []+ more += [_plural(notes, "note")] if notes else []+ if worst["effect"] == "advisory":+ grid.add_row(Text(f"{glyph} {row['harness']}", f"bold {color}"), Text(label, TEXT), ", ".join(more))+ continue+ grid.add_row(+ Text(f"{glyph} {row['harness']}", f"bold {color}"),+ Text.assemble((worst.get("outcome") or label, f"bold {color}"), (" ", ""), (worst["message"], TEXT)),+ ", ".join(more),+ )+ return grid if grid.row_count else None+++def _detail_grid(rows: list[dict[str, Any]], verbose: bool) -> Table | None:+ """Every finding with its fix, under a line naming the harness, its status and verified version."""+ grid = Table.grid(padding=(0, 2), expand=True)+ grid.add_column(no_wrap=True)+ grid.add_column(no_wrap=True)+ grid.add_column(ratio=1)+ for row in rows:+ if row["status"] in ("pending", "checking"):+ continue+ glyph, color, label, _ = _STATUS[row["status"]]+ version = f"verified {row.get('verified_version', '')}, {row.get('verified_date', '')}"+ grid.add_row(+ Text(f"{glyph} {row['harness']}", f"bold {color}"), Text(label, f"bold {color}"), Text(version, MUTED)+ )+ for finding in row.get("findings") or []:+ glyph, color, label = _EFFECT[finding["effect"]]+ grid.add_row(+ "",+ Text(f"{glyph} {label}", f"bold {color}" if finding["effect"] != "advisory" else MUTED),+ Text.assemble(+ (finding["message"], TEXT),+ (f" [{finding['id']}]", MUTED),+ *([("\n→ ", GREEN), (finding["fix"], MUTED)] if finding.get("fix") else []),+ ),+ )+ return grid if grid.row_count else None+++def _subtitle(rows: list[dict[str, Any]]) -> tuple[Text, str]:+ done = [row for row in rows if row["status"] not in ("pending", "checking")]+ if len(done) < len(rows):+ return Text(f"checking {len(done)}/{len(rows)}", MUTED), FAINT+ total = len(rows)+ for status, label, color in (("not_loaded", "not loaded", RED), ("degraded", "degraded", AMBER)):+ count = sum(row["status"] == status for row in rows)+ if count:+ glyph = _STATUS[status][0]+ return Text(f"{glyph} {count} of {total} {label}", f"bold {color}"), color+ return Text(f"✓ {total} of {total} load", f"bold {GREEN}"), GREEN+++def harness_panel(+ block: Block,+ *,+ title: str = "Harness compatibility",+ footer: str | None = FOOTER,+ verbose: bool = False,+ width: int = WIDTH,+) -> Panel:+ """One panel: a ✓/!/✗ line per skill, then its worst finding per harness (every finding with ``verbose``)."""+ parts: list[Any] = []+ many = len(block) > 1+ all_rows = [row for rows in block.values() for row in rows]+ for skill, rows in block.items():+ if parts:+ parts.append(Text())+ parts.append(_summary_line(rows, f"{Path(skill).name} " if many else ""))+ grid = _detail_grid(rows, verbose) if verbose else _compact_grid(rows)+ if grid is not None:+ parts.extend([Text(), grid])+ flagged = any(row.get("findings") for row in all_rows)+ if footer and (flagged or verbose):+ parts.extend([Text(), Text(footer + ("" if verbose else VERBOSE_HINT), MUTED)])+ subtitle, border = _subtitle(all_rows)+ return Panel(+ Group(*parts),+ box=box.ROUNDED,+ border_style=border,+ width=width,+ padding=(0, 2),+ title=Text(title, f"bold {TEXT}"),+ title_align="left",+ subtitle=subtitle,+ subtitle_align="right",+ )++def write_json_block(json_report: Path, block: Block) -> None:"""Add the ``harness_compatibility`` block to an existing JSON report."""if json_report.exists():data = json.loads(json_report.read_text(encoding="utf-8"))data[HARNESS_METADATA_KEY] = blockjson_report.write_text(json.dumps(data, indent=2), encoding="utf-8")+++def run_harness_check(+ target: Path,+ harness: str,+ profiles_dir: Path | None,+ *,+ console: Console,+ verbose: bool = False,+ config: dict[str, Any] | None = None,+) -> Block:+ """Evaluate each skill harness by harness, updating a live panel as each finishes."""+ target = target.parent if target.is_file() else target+ profiles = load_profiles(harness, profiles_dir)+ finder = HarnessValidator()+ skills = [target] if finder._is_skill_directory(target) else finder._find_all_skills(target)+ width = max(60, min(WIDTH, console.width))+ block: Block = {}+ all_facts = facts_for(skills)+ for skill in skills:+ rows: list[dict[str, Any]] = [{"harness": p["harness"], "status": "pending"} for p in profiles]++ def panel(rows: list[dict[str, Any]] = rows, skill: Path = skill) -> Panel:+ return harness_panel(+ {str(skill): rows},+ title=f"Harness compatibility · {skill.name}",+ verbose=verbose,+ width=width,+ )++ with Live(panel(), console=console, refresh_per_second=12) as live:+ facts = all_facts[skill]+ for index, profile in enumerate(profiles):+ rows[index] = {"harness": profile["harness"], "status": "checking"}+ live.update(panel())+ rows[index] = evaluate_profile(profile, facts, config).to_dict()+ live.update(panel())+ console.print()+ block[str(skill)] = rows+ return blockdiff --git a/src/skillevaluator/reporting/console_ui.py b/src/skillevaluator/reporting/console_ui.pyindex ef6541b0..7c3ffc3c 100644--- a/src/skillevaluator/reporting/console_ui.py+++ b/src/skillevaluator/reporting/console_ui.py@@ -103,6 +103,7 @@ RUNNING = "running"PASS = "pass"FAIL = "fail"SKIP = "skip"+WARN = "warn"def _fmt_duration(seconds: float | None) -> str:@@ -554,6 +555,8 @@ class ValidateView:self.verdict: Verdict | None = Noneself.links: list[tuple[str, str]] = []self._live: Live | None = None+ self.harness_panel: Panel | None = None+ self.harness_note: str | None = Noneself.progress_reporter = Noneself._progress_stage: dict[int, set[str]] = {}self._static_printed = 0 # blocks already flushed in non-tty mode@@ -641,6 +644,12 @@ class ValidateView:self._flush_static()self._refresh()+ def set_harness(self, panel: Panel, note: str | None) -> None:+ """Show the advisory harness panel after the Tier 1 block and *note* in the verdict."""+ self.harness_panel = panel+ self.harness_note = note+ self._refresh()+def finish(self, verdict: Verdict, links: list[tuple[str, str]]) -> None:if not self.enabled:return@@ -683,10 +692,12 @@ class ValidateView:if len(self.blocks) >= 2:yield self._gate()yield Text()- for block in self.blocks:+ for index, block in enumerate(self.blocks):if block.status == PENDING:continueyield self._block_panel(block)+ if index == 0 and self.harness_panel is not None:+ yield self.harness_panelif self.verdict is not None:yield self._verdict_panel(self.verdict)yield Text()@@ -807,10 +818,16 @@ class ValidateView:def _verdict_panel(self, verdict: Verdict) -> Panel:headline = strip_terminal_controls(verdict.headline)+ note = (+ Text.assemble(("! harness: ", f"bold {AMBER}"), (strip_terminal_controls(self.harness_note), AMBER))+ if self.harness_note+ else None+ )if verdict.passed:body = Text.assemble(_pill("✓ PASS", GREEN), (" ", ""), (headline, f"bold {TEXT}"), (" · exit 0", MUTED))+ details = [Text(strip_terminal_controls(line), style=MUTED) for line in verdict.details]return Panel(- Group(body, *(Text(strip_terminal_controls(line), style=MUTED) for line in verdict.details)),+ Group(body, *details, *([Text(), note] if note is not None else [])),box=box.ROUNDED,border_style=GREEN,width=self.width,@@ -824,6 +841,8 @@ class ValidateView:(" · exit 1", MUTED),)parts: list = [line1, *(Text(strip_terminal_controls(line), style=MUTED) for line in verdict.details)]+ if note is not None:+ parts.extend([Text(), note])if verdict.fix:fix = Text.assemble(("fix ", f"bold {MUTED}"))for chunk, style in verdict.fix:@@ -853,9 +872,12 @@ class ValidateView:breakdone.append(block)# Advance first so the live region drops these blocks as they print above it.+ first = self._static_printedself._static_printed += len(done)- for block in done:+ for index, block in enumerate(done, start=first):self.console.print(self._block_panel(block))+ if index == 0 and self.harness_panel is not None:+ self.console.print(self.harness_panel)def check_ticker_row(lineup: list[str], done: list[str], current: str | None) -> TierRow:@@ -1079,7 +1101,7 @@ def final_check_ticker_row(lineup: list[str], results: list[ValidationResult]) -if not result.passed:status[key] = FAILelif status.get(key) != FAIL:- status[key] = PASS+ status[key] = WARN if _harness_flagged(result) else PASSsegments: list[tuple[str, str]] = []for i, name in enumerate(lineup):if i:@@ -1087,6 +1109,8 @@ def final_check_ticker_row(lineup: list[str], results: list[ValidationResult]) -state = status.get(name)if state == FAIL:segments.append((f"✗ {name}", f"bold {RED}"))+ elif state == WARN:+ segments.append((f"! {name}", f"bold {AMBER}"))elif state == PASS:segments.append((f"✓ {name}", GREEN))elif name in user_skipped:@@ -1096,6 +1120,12 @@ def final_check_ticker_row(lineup: list[str], results: list[ValidationResult]) -return TierRow("checks", segments)+def _harness_flagged(result: ValidationResult) -> bool:+ """True when the advisory harness check found a harness that drops or degrades a skill."""+ block = (result.metadata or {}).get("harness_compatibility") or {}+ return any(row.get("status") != "loads" for rows in block.values() for row in rows)++def _short_name(validator_name: str) -> str:name = (validator_name or "check").strip()aliases = {@@ -1195,6 +1225,19 @@ def summarize_tier1(results: list[ValidationResult], lineup: list[str] | None =if advisory:detail = " · ".join(f"{name} {count}" for name, count in sorted(advisory.items(), key=lambda kv: -kv[1]))rows.append(TierRow("advisories", [(str(sum(advisory.values())), TEXT), (f" {detail}", MUTED)]))+ harness_rows = [+ row+ for r in results+ for rows in ((r.metadata or {}).get("harness_compatibility") or {}).values()+ for row in rows+ ]+ counts = [+ f"{sum(row.get('status') == status for row in harness_rows)} {label}"+ for status, label in (("not_loaded", "not loaded"), ("degraded", "degraded"))+ if any(row.get("status") == status for row in harness_rows)+ ]+ if counts:+ rows.append(TierRow("harness", [(" · ".join(counts), TEXT), (" (see below)", MUTED)]))suppressed_security = 0for r in results:diff --git a/tests/golden/cli_surface.json b/tests/golden/cli_surface.jsonindex bbf6e91d..e5a902fc 100644--- a/tests/golden/cli_surface.json+++ b/tests/golden/cli_surface.json@@ -1281,6 +1281,55 @@}]},+ "harness-check": {+ "params": [+ {+ "name": "target_path",+ "opts": [+ "target_path"+ ],+ "param_type": "argument",+ "required": true,+ "type": "path"+ },+ {+ "default": "all",+ "name": "harness",+ "opts": [+ "--harness"+ ],+ "param_type": "option",+ "type": "text"+ },+ {+ "name": "harness_profiles",+ "opts": [+ "--harness-profiles"+ ],+ "param_type": "option",+ "type": "directory"+ },+ {+ "default": "False",+ "is_flag": true,+ "name": "verbose",+ "opts": [+ "--verbose",+ "-v"+ ],+ "param_type": "option",+ "type": "boolean"+ },+ {+ "name": "json_path",+ "opts": [+ "--json"+ ],+ "param_type": "option",+ "type": "file"+ }+ ]+ },"health-check": {"params": [{diff --git a/tests/harness/test_overlay.py b/tests/harness/test_overlay.pyindex a73650b4..7127852d 100644--- a/tests/harness/test_overlay.py+++ b/tests/harness/test_overlay.py@@ -11,8 +11,10 @@ from click.testing import CliRunnerfrom skillevaluator.cli import clifrom skillevaluator.harness import HARNESS_STATUSES, evaluate_harnesses, load_profiles, skill_factsfrom skillevaluator.harness.overlay import PROFILES_DIR+from skillevaluator.harness.report import harness_block, harness_note, harness_panelfrom skillevaluator.models.result import Severity-from skillevaluator.tier1.commands import DEFAULT_CHECKS+from skillevaluator.reporting.console_ui import final_check_ticker_row, summarize_tier1+from skillevaluator.tier1.commands import DEFAULT_CHECKS, run_validationfrom skillevaluator.validators.harness import HarnessValidatorHARNESSES = ["claude-code", "codex", "copilot-cli", "hermes", "openclaw", "opencode"]@@ -187,6 +189,14 @@ def test_cli_rejects_unknown_harness(tmp_path: Path):assert "Unknown harness: nope" in result.output+def _render(panel) -> str:+ from rich.console import Console++ console = Console(width=120, record=True, color_system=None)+ console.print(panel)+ return console.export_text()++def test_harness_check_runs_by_default_and_never_fails(tmp_path: Path):assert "harness" in DEFAULT_CHECKSresult = HarnessValidator("claude-code,codex").validate(_rooted(tmp_path))@@ -200,6 +210,70 @@ def test_harness_check_runs_by_default_and_never_fails(tmp_path: Path):assert {r["harness"]: r["status"] for r in rows} == {"claude-code": "not_loaded", "codex": "loads"}+def test_harness_check_alone_marks_ticker_and_panel(tmp_path: Path):+ results = run_validation(_rooted(tmp_path), checks="harness", harness="all")+ block = harness_block(results)++ assert [r.validator_name for r in results] == ["Harness Compatibility"]+ assert any(text == "! harness" for text, _ in final_check_ticker_row(["harness"], results).segments)+ assert harness_note(block) == "skill won't load in claude-code"+ text = _render(harness_panel(block))+ assert "Harness compatibility" in text and "advisory" not in text+ assert "not found" in text and "[CC-ROOT]" not in text+ assert "1 of 6 not loaded" in text and "Details and fixes: -v" in text+ full = _render(harness_panel(block, verbose=True))+ assert "NOT LOADED" in full and "[CC-ROOT]" in full+++def test_clean_skill_panel_is_one_line(tmp_path: Path):+ block = harness_block(run_validation(_skill(tmp_path), checks="harness"))++ assert harness_note(block) is None+ text = _render(harness_panel(block))+ assert "6 of 6 load" in text and "harness-check" not in text+++def test_validate_shows_panel_and_verdict_note(tmp_path: Path):+ skill = _rooted(tmp_path)+ result = CliRunner().invoke(+ cli, ["validate", str(skill), "--tiers", "1", "--checks", "schema,harness", "-o", str(tmp_path / "r")]+ )++ assert result.exit_code == 0, result.output+ assert "Harness compatibility" in result.output and "· advisory" not in result.output+ assert "! harness: skill won't load in claude-code" in result.output+++def test_harness_check_command_exits_zero_and_writes_json(tmp_path: Path):+ skill = _rooted(tmp_path)+ out = tmp_path / "h.json"+ result = CliRunner().invoke(+ cli, ["harness-check", str(skill), "--harness", "codex,claude-code", "--json", str(out)]+ )++ assert result.exit_code == 0, result.output+ assert "Harness compatibility · demo-skill" in result.output+ assert "1 of 2 not loaded" in result.output+ rows = next(iter(json.loads(out.read_text())["harness_compatibility"].values()))+ assert [r["status"] for r in rows] == ["loads", "not_loaded"]+++def test_harness_check_command_rejects_unknown_harness(tmp_path: Path):+ result = CliRunner().invoke(cli, ["harness-check", str(_skill(tmp_path)), "--harness", "nope"])++ assert result.exit_code == 2+ assert "Unknown harness: nope" in result.output+++def test_tier1_summary_gives_harness_its_own_line(tmp_path: Path):+ results = run_validation(_rooted(tmp_path), checks="schema,harness", harness="all")+ _, rows = summarize_tier1(results)+ labels = {row.label: "".join(text for text, _ in row.segments) for row in rows}++ assert labels["harness"].startswith("1 not loaded")+ assert "harness" not in labels.get("advisories", "")++def test_native_root_skips_root_rules(tmp_path):skill = tmp_path / ".claude" / "skills" / "demo"skill.mkdir(parents=True)@@ -210,3 +284,12 @@ def test_native_root_skips_root_rules(tmp_path):assert "CX-ROOT" in ids({})assert not {i for i in ids({"native_root": True}) if i.endswith("-ROOT")}+++def test_harness_note_names_the_skill_not_the_harness():+ block = {+ "a": [{"harness": "codex", "status": "not_loaded"}, {"harness": "hermes", "status": "degraded"}],+ "b": [{"harness": "opencode", "status": "not_loaded"}, {"harness": "hermes", "status": "loads"}],+ }++ assert harness_note(block) == "2 skills won't load in codex, opencode · skill loads degraded in hermes"