Skip to content

The agent that grows with you

plugin-catalog: point skill-lineage at 37132da and add the evaluation result

Simin (Simon) Yuan451bc9a33d22
diff --git a/plugin-catalog/skill-lineage.yaml b/plugin-catalog/skill-lineage.yaml
index 4fe4ad8987d..09b6ed063dd 100644
--- a/plugin-catalog/skill-lineage.yaml
+++ b/plugin-catalog/skill-lineage.yaml
@@ -1,7 +1,7 @@
name: skill-lineage
title: Skill Lineage
repo: https://github.com/simin-yuan/hermes-skill-lineage
-sha: cbb85a979e70622768c195dbfe5d8b7cbcaaa3d1
+sha: 37132da4b8fb8935f4ef07430bc854cf2bcc680c
version: "0.1.0"
description: >-
Records where each skill version's content came from, so a skill that rewrites itself can be audited
@@ -21,8 +21,15 @@ description: >-
shown in `status` rather than happening silently. The plugin does not stop a write and does not judge
a skill: it records and it verifies, and it does not claim a source caused an edit. Prior work on
memory lineage (MemLineage, arXiv:2605.14421) is acknowledged in the README.
- Measured on 2026-10-21 against Hermes 3166d61: Python 3.14 56 passed; Python 3.11 48 passed, 1 skipped;
- `hermes plugins validate` passes with no warnings. A manual end-to-end run in an isolated HERMES_HOME
+ Measured on 2026-10-21 against Hermes 3166d61: Python 3.14 59 passed; Python 3.11 51 passed,
+ 1 skipped; `hermes plugins validate` reports no warning other than one known false positive (a
+ test template that quotes a "curl | sh" instruction as data and never runs it).
+ A labelled synthetic evaluation — 600 generated scenarios on a held-out seed, with the acceptance
+ floors fixed before the run — puts precision at 0.988 for the default link set, with recall 1.00
+ on verbatim, paraphrase-with-operative-token and two-hop-via-curator cases, against 0.385 when
+ every link is recorded; the README states plainly that the paraphrases keep their operative token
+ by construction and that a rewrite by a real model is not yet measured. Raw output in
+ eval/results-seed2.json, reproducible with `python eval/run_eval.py --seed 2 --per-label 100`. A manual end-to-end run in an isolated HERMES_HOME
(one real session creating a skill through skill_manage, then
status/trace/descendants/report/verify/head) passed, raw output in tests/smoke-2026-10-21.txt.
Disclosure — four read-only hooks (on_session_start, pre_llm_call, pre_tool_call, post_tool_call);