Skip to content

armgit / ruflo

Public Imported from GitHub
0

🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated

docs(adr): ADR-152 β€” Genome Similarity Search (Phase 3 Β§3.1 critical path, iter 34)

ruv06852f991b7c
diff --git a/plugins/ruflo-metaharness/scripts/smoke.sh b/plugins/ruflo-metaharness/scripts/smoke.sh
index 3a4428e317..4f47247988 100755
--- a/plugins/ruflo-metaharness/scripts/smoke.sh
+++ b/plugins/ruflo-metaharness/scripts/smoke.sh
@@ -191,6 +191,26 @@ grep -q "execCli(\[\s*'-y'\s*,\s*'metaharness@latest'" "$F" 2>/dev/null || \
grep -q "cwd: opts" "$F" || miss="$miss no-cwd-passthrough"
[[ -z "$miss" ]] && ok || bad "$miss"
+step "17w. ADR-152 Genome Similarity Search drafted (iter 34, Phase 3 critical-path)"
+F="$ROOT/../../v3/docs/adr/ADR-152-genome-similarity-search.md"
+miss=""
+[[ -f "$F" ]] || miss="$miss adr-missing"
+# Must reference its parent
+grep -q "ADR-151" "$F" 2>/dev/null || miss="$miss no-parent-link"
+# Must enumerate the 9 numerical features used in the cosine
+for field in harnessFit compileConfidence taskCoverage toolSafety memoryUsefulness risk_score test_confidence publish_readiness estCostPerRunUsd; do
+ grep -q "$field" "$F" 2>/dev/null || miss="$miss missing-feature-${field}"
+done
+# Composite weights documented
+grep -q "0.6.*cosine.*0.25.*categorical.*0.15.*jaccard" "$F" 2>/dev/null || miss="$miss no-weights"
+# Smallest-spike contract present
+grep -q "Smallest demonstrable spike" "$F" 2>/dev/null || miss="$miss no-spike-contract"
+# Cross-link from ADR-151 updated to DRAFTED
+PARENT151="$ROOT/../../v3/docs/adr/ADR-151-harness-intelligence-layer.md"
+grep -q "ADR-152-genome-similarity-search.md" "$PARENT151" 2>/dev/null || miss="$miss adr151-not-updated"
+grep -q "DRAFTED iter 34" "$PARENT151" 2>/dev/null || miss="$miss adr151-no-drafted-marker"
+[[ -z "$miss" ]] && ok || bad "$miss"
+
step "17v. ADR-151 Phase 3 scope shell drafted (iter 33)"
F="$ROOT/../../v3/docs/adr/ADR-151-harness-intelligence-layer.md"
miss=""
diff --git a/v3/docs/adr/ADR-151-harness-intelligence-layer.md b/v3/docs/adr/ADR-151-harness-intelligence-layer.md
index 9778e942d0..8b2080a7a1 100644
--- a/v3/docs/adr/ADR-151-harness-intelligence-layer.md
+++ b/v3/docs/adr/ADR-151-harness-intelligence-layer.md
@@ -31,7 +31,7 @@ Adopt the **Harness Intelligence Layer** as a Phase-3 capability cluster, scoped
### Phase-3 sub-capabilities (in implementation-dependency order)
-#### 3.1 β€” Genome Similarity Search (ADR-152, to be drafted)
+#### 3.1 β€” Genome Similarity Search ([ADR-152](ADR-152-genome-similarity-search.md), DRAFTED iter 34)
**What:** Given two harnesses (or one harness + a candidate repo), compute a similarity score across the 7 genome sections and the 5 scorecard dimensions. Return a normalized 0..1 number plus a per-dimension breakdown.
diff --git a/v3/docs/adr/ADR-152-genome-similarity-search.md b/v3/docs/adr/ADR-152-genome-similarity-search.md
new file mode 100644
index 0000000000..bdbd0beaf1
--- /dev/null
+++ b/v3/docs/adr/ADR-152-genome-similarity-search.md
@@ -0,0 +1,161 @@
+# ADR-152 β€” Genome Similarity Search
+
+**Status**: Proposed
+**Date**: 2026-06-16
+**Parent**: [ADR-151](ADR-151-harness-intelligence-layer.md) (Phase 3 scope shell β€” Harness Intelligence Layer)
+**Inherits**: ADR-150's four architectural constraints (removable / optional / graceful / CI-gate)
+
+## Context
+
+ADR-151 Phase-3 Β§3.1 calls for a similarity function over MetaHarness genome + scorecard JSON. It is the **critical-path dependency for Β§3.2 (Recommendation Engine), Β§3.3 (Fleet Drift), and Β§3.5 (Plugin Compatibility)** β€” three of the four other Phase-3 sub-capabilities consume similarity scores.
+
+What we have already (post-ADR-150 implementation):
+- `harness genome <repo>` emits a 7-section JSON blob: `repo_type`, `agent_topology[]`, `risk_score`, `mcp_surface`, `test_confidence`, `publish_readiness`.
+- `harness score <repo>` emits a 5-dimension JSON blob: `harnessFit`, `compileConfidence`, `taskCoverage`, `toolSafety`, `memoryUsefulness`, plus `archetype` and `template`.
+- `harness threat-model <repo>` emits a `worst` severity + a list of categorized findings.
+- The iter-7 `oia-audit` records bundle all three above per timestamp in `metaharness-audit` memory namespace.
+
+What we need: a function `similarity(genomeA, genomeB) β†’ number ∈ [0,1]` plus a per-dimension breakdown explaining where the two harnesses agree or differ.
+
+## Decision
+
+Implement genome similarity as a **pure-TS function** in the existing `ruflo-metaharness` plugin with three components:
+
+### 1. Cosine similarity over a numerical feature vector
+
+Project the structured genome + scorecard into a fixed-length numerical vector:
+
+| Index | Source | Field | Normalization |
+|---:|---|---|---|
+| 0 | score | `harnessFit` | already 0..100 β†’ divide by 100 |
+| 1 | score | `compileConfidence` | divide by 100 |
+| 2 | score | `taskCoverage` | divide by 100 |
+| 3 | score | `toolSafety` | divide by 100 |
+| 4 | score | `memoryUsefulness` | divide by 100 |
+| 5 | genome | `risk_score` | already 0..1 |
+| 6 | genome | `test_confidence` | already 0..1 |
+| 7 | genome | `publish_readiness` | already 0..1 |
+| 8 | score | `estCostPerRunUsd` | log-transformed: `log10(usd + 0.001) / log10(10)` clamped to 0..1 |
+
+Cosine similarity over these 9 dims gives a `[0, 1]` score where 1 = identical and 0 = orthogonal. Cheap, deterministic, byte-identical for identical inputs.
+
+### 2. Categorical agreement over enum fields
+
+Three fields are categorical:
+
+| Field | Source | Possible values |
+|---|---|---|
+| `repo_type` | genome | e.g. `node_mcp_ci`, `python_lib`, `rust_cli` |
+| `archetype` | score | e.g. `typescript-sdk-harness`, `python-agent-harness` |
+| `template` | score | one of the 20 metaharness templates |
+| `recommendedMode` | score | `CLI`, `MCP`, or `CLI + MCP` |
+
+Each contributes `1` if matching, `0` if not. Sum / 4 β†’ categorical-agreement score.
+
+### 3. Set agreement over `agent_topology[]`
+
+The `agent_topology` field is an array (e.g. `["maintainer", "tester", "security", "release"]`). Jaccard similarity = `|A ∩ B| / |A βˆͺ B|`.
+
+### Composite
+
+`overallSimilarity = 0.6 Β· cosine + 0.25 Β· categorical + 0.15 Β· jaccard`
+
+Weights chosen so that:
+- Numerical similarity dominates (most signal density).
+- Categorical agreement is the next tier (high-signal but low-cardinality).
+- Topology agreement is the tie-breaker (interpretable but coarse).
+
+Return shape:
+
+```typescript
+interface SimilarityResult {
+ overall: number; // [0,1]
+ components: {
+ cosine: number; // [0,1]
+ categorical: number; // [0,1]
+ jaccard: number; // [0,1]
+ };
+ perDimension: Record<string, {
+ a: number | string | string[];
+ b: number | string | string[];
+ contribution: number; // signed [-w, +w]
+ }>;
+ generatedAt: string;
+}
+```
+
+The `perDimension` breakdown lets consumers explain *why* two harnesses scored as they did β€” critical for the Β§3.2 Recommendation Engine's confidence calculation and the Β§3.3 Drift Detection's alert reason.
+
+## Implementation surface
+
+- **One pure-TS function** in `plugins/ruflo-metaharness/scripts/_similarity.mjs` (shared module convention β€” see iter-1 `_harness.mjs` and iter-73 `_sessions.mjs`).
+- **One CLI skill** `harness-similarity` invoked as `npx ruflo metaharness similarity --a <genomeA.json> --b <genomeB.json>` (or `--a-key`/`--b-key` for memory-namespace lookup, mirroring iter-15 `audit-trend`).
+- **One MCP tool** `mcp__claude-flow__metaharness_similarity` so agents can call it during conversation.
+
+NO new dependency on `@metaharness/*` β€” the function operates on JSON shapes that the existing CLI already emits. Genuinely zero blast radius on ADR-150's four constraints:
+
+- Removable βœ“ (pure-TS, reads JSON the CLI already produces)
+- Optional βœ“ (no new dep)
+- Graceful βœ“ (gracefully reports `degraded` if inputs are malformed)
+- CI-gate βœ“ (standalone unit-testable; no `npx` required)
+
+## Smallest demonstrable spike (the implementation gate)
+
+Before ADR-152 is marked **Accepted**, ship a 30-LOC proof that:
+
+1. Reads two `harness genome` + `harness score` JSON files from disk.
+2. Computes `cosine` over the 9 numeric dims.
+3. Returns a single number.
+4. Verifies: similarity(X, X) === 1 (exact self-match).
+5. Verifies: similarity(legal_harness, devops_harness) < similarity(legal_harness, support_harness) using the 13 known metaharness verticals as fixtures.
+
+The spike script: `plugins/ruflo-metaharness/scripts/_spike-similarity.mjs`. Lives in the plugin to avoid polluting global scripts; deleted after ADR-152 graduates.
+
+## Consequences
+
+### Positive
+
+- Unblocks Β§3.2 / Β§3.3 / Β§3.5 β€” three of four Phase-3 consumers can begin work in parallel once similarity ships.
+- No new dependencies, no runtime cost, pure-function semantics. Failure modes are limited to "garbage in β†’ low confidence out" β€” never a crash.
+- The 9-dim feature vector + 3-component weighted composite is a **single source of truth** for similarity semantics; downstream ADRs reference back to this one rather than reinventing.
+
+### Negative
+
+- **Choice of weights (0.6/0.25/0.15) is opinionated.** A different weighting could change the recommender's rankings. Mitigation: expose weights as optional CLI flags; defaults are documented; ship the fixtures from the spike as a regression-suite so weight changes are visible.
+- **Numerical vector excludes `threat-model worst`** because it's categorical and would inflate the categorical component too far. If `worst` regression is what matters for drift detection (Β§3.3), drift consumers must check `worst` separately.
+- **No semantic similarity over agent prompts / SKILL.md text.** ADR-151's Β§3.4 (capability graph) calls for that; this ADR explicitly defers it to keep Β§3.1 small.
+
+### Neutral
+
+- The function is invocable from any layer (CLI / MCP tool / direct Node import / future browser-side) without changes β€” the JSON contract is the only coupling.
+
+## Alternatives Considered
+
+**Alternative A: Use sentence embeddings over the README + agent prompt text.**
+- Pros: captures semantic similarity humans care about.
+- Cons: requires embeddings model + 384-dim vectors + similarity index. Adds a heavy dep, breaks ADR-150's removability rule for harnesses that disable optional deps.
+- Verdict: Defer to Β§3.4 (capability graph) where it's actually needed.
+
+**Alternative B: Skip cosine, use only categorical + jaccard.**
+- Pros: simpler, no float math.
+- Cons: loses signal from the 5 scorecard + 3 genome numeric dimensions. `harnessFit: 82` vs `harnessFit: 45` should clearly contribute to similarity, and categorical can't see that.
+- Verdict: cosine over the 9 numerics is the right primitive.
+
+**Alternative C: Train a similarity model on real harness pairs.**
+- Pros: would learn actual user-perceived similarity.
+- Cons: requires labeled pairs that don't exist; circular (the recommender's job is to PRODUCE those pairs). Premature ML.
+- Verdict: defer until Β§3.2 has shipped enough data to bootstrap.
+
+## Open Questions
+
+- Should weights be a configurable field in `mcp-policy.json`? Lean no β€” the 0.6/0.25/0.15 split should be a single global default. Per-org tuning is out of scope for Β§3.1.
+- Should the function ALSO emit a Mermaid diagram visualizing the breakdown? Probably yes via an optional `--format mermaid`. Cheap to add post-spike.
+- How does similarity interact with the iter-15 `audit-trend` script? `audit-trend` already diffs two timestamps of the SAME harness; this ADR adds a separate primitive for diffing two DIFFERENT harnesses. Both should live as siblings, not subtypes.
+
+## References
+
+- [ADR-151](ADR-151-harness-intelligence-layer.md) β€” Phase 3 scope shell (parent)
+- [ADR-150](ADR-150-metaharness-integration-surfaces.md) β€” Phase 1+2 implementation, architectural constraints
+- iter-15 `audit-trend` β€” `plugins/ruflo-metaharness/scripts/audit-trend.mjs` (sibling drift-detection primitive)
+- iter-1 `_harness.mjs` β€” shared-module convention reference
+- Upstream `harness genome / score / threat-model` outputs documented in [`ruvnet/agent-harness-generator`](https://github.com/ruvnet/agent-harness-generator) source