Skip to content

armgit / ruflo

Public Imported from GitHub
0

🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated

docs(adr): ADR-150 β€” MetaHarness integration surfaces (Proposed)

ruv04826eaf3f9f
diff --git a/v3/docs/adr/ADR-150-metaharness-integration-surfaces.md b/v3/docs/adr/ADR-150-metaharness-integration-surfaces.md
new file mode 100644
index 0000000000..e367b7dd66
--- /dev/null
+++ b/v3/docs/adr/ADR-150-metaharness-integration-surfaces.md
@@ -0,0 +1,129 @@
+# ADR-150 β€” MetaHarness Integration Surfaces in `npx ruflo`
+
+**Status**: Proposed
+**Date**: 2026-06-16
+**Related**: ADR-148 (cost-optimal router lifecycle via `@metaharness/router`), ADR-149 (per-model cost-optimal routing), ADR-026 (3-tier model routing), ADR-097 (federation budget circuit breaker), ADR-124 (optional native dependencies), ADR-144 (agent-authorization-propagation)
+**External reference**: [`ruvnet/agent-harness-generator`](https://github.com/ruvnet/agent-harness-generator) β€” the upstream that publishes `metaharness` + `@metaharness/*`. Same author (rUv), explicitly designed around ruflo primitives.
+**Research dossier**: published as a gist (linked from the tracking issue) with full graded-evidence sourcing.
+
+## Context
+
+We just shipped `[email protected]` (also `@claude-flow/[email protected]`, `[email protected]`). ADR-148/149 already wired `@metaharness/router` as an `optionalDependency` for cost-optimal model routing behind a triple gate. The remaining MetaHarness surface β€” twenty-plus `@metaharness/*` packages: kernel, host adapters (9), verticals (13), scaffold/eject CLI β€” is unused by ruflo despite being authored by the same maintainer specifically around ruflo's architecture.
+
+Three signals make this the right time to commit a broader integration:
+
+1. **MetaHarness is first-party.** Same author (`[email protected]`), same ADR numbering convention (kernel docs reference ADR-011/022/033/036/040/041/043), explicit framing: *"Scaffold your own focused AI agent harness β€” like ruflo, uniquely yours."* The `buildRegistryEntry()` doc comment says: *"Mirrors the ruflo plugin registry shape so the same UI can browse it."* The `@metaharness/host-claude-code` adapter emits `.claude/settings.json` in exactly ruflo's format.
+2. **The router integration is already live but underutilized.** `@metaharness/router@^0.3.2` is in `optionalDependencies`; `neural-router.ts` imports it behind `CLAUDE_FLOW_ROUTER_NEURAL=1`. The bundled KRR is trained on hand-coded seed scores rather than measured routing outcomes β€” leaving the DRACO Pareto win unrealized.
+3. **No ruflo skill exposes scaffolding/score/genome/threat-model to Claude Code today.** Users discover MetaHarness independently and are confused about the relationship.
+
+### Evidence baseline (measured 2026-06-16)
+
+| Fact | Source | Grade |
+|---|---|---|
+| `[email protected]` ships 24 subcommands across two binaries (`metaharness` factory + `harness` lifecycle) | `dist/index.d.ts`, `dist/subcommands.d.ts`, all `*-cmd.d.ts` | HIGH |
+| 20+ `@metaharness/*` packages published; full ecosystem (kernel + 5 host adapters + 13 verticals + 5 platform NAPI binaries) | `npm search @metaharness` | HIGH |
+| `@metaharness/[email protected]` exports `Router` (k-NN), `TrainedRouter` (KRR), `NativeRouter` (FastGRNN via tiny-dancer), zero runtime deps, 53 kB unpacked | `dist/*.d.ts`, npm registry | HIGH |
+| `@metaharness/[email protected]` exports `loadKernel`, `ToolDispatcher` (claims-checked), `SelfEvolvingRouter`, `TrajectoryStore`, `rankWithDecay` | `kernel-pkg/package/dist/*.d.ts` | HIGH |
+| `metaharness` factory exports `buildRepoScorecard()`, `buildGenomeReport()`, `buildScorecard()`, `buildThreatModel()`, `scanMcp()`, `buildOiaManifest()`, `buildRegistryEntry()` β€” all pure reads, well-typed | `dist/repo-scorecard.d.ts` etc. | HIGH |
+| Velocity: `metaharness` 0.1.0 β†’ 0.1.11 in ~23h; `@metaharness/router` 0.1.0 β†’ 0.3.2 in 2.7h on 2026-06-15 | npm `time` field | HIGH |
+| Both packages MIT-licensed, same maintainer as ruflo | npm registry | HIGH |
+| Existing benchmark proves `@metaharness/router` native backend loads on the test host: `mh_native_available: true` | `docs/benchmarks/runs/router-4way-seed99-2026-06-15T14-12-40Z.json` | HIGH |
+
+## Decision
+
+Adopt MetaHarness as ruflo's downstream sibling tool, surfaced through three integration channels that match its three distinct contributions:
+
+1. **Static-analysis MCP tools** β€” `harness-score`, `harness-genome`, `harness-threat-model`, `harness-mcp-scan` as a new `plugins/ruflo-metaharness/` plugin. Subprocess invocation of the `metaharness` / `harness` CLI binaries; no static library dependency added to ruflo's boot path. Read-only operations only.
+2. **Live router data pipeline** β€” replace the hand-coded seed corpus for the bundled KRR with measured routing trajectories collected via the existing `CLAUDE_FLOW_ROUTER_TRAJECTORY=1` recorder; retrain `train-bundled-krr.mjs` against real data. This unlocks the Pareto win ADR-149 forecast but never measured.
+3. **CI security gates** β€” add `harness mcp scan .` and `metaharness score . --json` to `v3-ci.yml`. Both are static, fast, and machine-readable. Asserts no HIGH MCP findings and a non-zero readiness score on every PR.
+
+Three concrete things we ARE NOT doing in this ADR (deferred to Phase 2+):
+
+- Wiring `@metaharness/kernel`'s `ToolDispatcher` into the MCP dispatch core. The kernel is v0.1.0 and the dispatch path is too high-blast-radius for an early-stage replacement.
+- Promoting `@metaharness/router` from `optionalDependency` to `dependency`. The triple gate is the right posture until the API stabilizes at 1.0.
+- Exposing `from-repo <git-url>` as an MCP tool callable by Claude Code without explicit user confirmation. Untrusted-Git-clone is a deliberate human-in-the-loop step.
+
+### Phased rollout
+
+**Phase 0 β€” Measurement spike (1–3 days, no code shipped to npm).**
+- Run `npx metaharness score .` and `npx metaharness genome .` against the ruflo repo to establish baseline scorecards.
+- Enable `CLAUDE_FLOW_ROUTER_TRAJECTORY=1` for β‰₯50 routing decisions; verify the `.swarm/model-router-trajectories.jsonl` shape matches what `train-bundled-krr.mjs` expects.
+- Confirm `import('@metaharness/router')` succeeds from `v3/@claude-flow/cli` and exercise `Router.fromExamples(...)` with the existing benchmark corpus.
+- Run `harness mcp scan .` to baseline ruflo's own MCP threat-model score.
+
+Exit criteria: baseline numbers in hand; no surprises in trajectory format or `mcp scan` output.
+
+**Phase 1 β€” MVP (3–7 days, one MINOR release: 3.12.0).**
+1. **`plugins/ruflo-metaharness/`** with three skills (`harness-score`, `harness-genome`, `harness-mint`), conventional structure (`plugin.json`, `skills/*/SKILL.md` with `allowed-tools: Bash`, `scripts/smoke.sh`). Skills shell out to `npx metaharness` / `npx harness` β€” no library imports. Covered by the fleet meta-smoke and the three existing audits (exit-bypass, frontmatter, manifest).
+2. **CI gates** in `v3-ci.yml`: `npx metaharness score . --json` (assert `exitCode === 0`) and `npx harness mcp scan .` (assert no HIGH findings). Both are additive jobs on the existing matrix.
+3. **Real seed corpus**: collect trajectory data over Phase-0's recorder runs + a CI pass, retrain via `scripts/train-bundled-krr.mjs`, regenerate the bundled artifact. Validate `routedBy: 'metaharness-krr'` activates on real decisions in the next bench run.
+
+Exit criteria: `plugins/ruflo-metaharness/scripts/smoke.sh` passes; meta-smoke shows 33/33 plugins green; CI score + mcp-scan jobs green on main; new bench run shows `routedBy: 'metaharness-krr'` for β‰₯ 1 routing decision driven by measured-seed KRR.
+
+Semver: MINOR β€” additive plugin, additive CI gates, additive MCP tools. No breaking changes.
+
+**Phase 2 β€” Expansion (1–4 weeks, one or two MINOR releases).**
+- `npx ruflo eject` command wrapping `metaharness --from-existing ./` for one-shot harness extraction (attribution preserved via the `<!-- ruflo-attribution-block -->` convention).
+- `SelfEvolvingRouter` (from `@metaharness/kernel`) parallel-logged alongside the Thompson bandit in `model-router.ts` for two weeks; promote to default only if disagreement < 5% or quality improvement > 2%.
+- Harness entries in the ruflo plugin registry β€” accept `type: 'harness'` in `discovery.ts`; surface via `npx ruflo plugins list --type harness`.
+- 13th background worker `oia-audit` that runs `buildOiaManifest()` + `buildThreatModel()` + `scanMcp()` on a schedule and stores results in the `metaharness-audit` memory namespace.
+
+Each Phase-2 item is independently scoped and can ship as separate MINOR releases.
+
+## Consequences
+
+### Positive
+
+- Closes the "what is the relationship between ruflo and MetaHarness?" question by answering it in the UX rather than the docs.
+- Ruflo gains a continuous, machine-readable readiness score and MCP threat-model on every PR β€” the same primitives we use to score third-party repos for harness viability.
+- The ADR-149 Pareto win (per-model cost-optimal routing) becomes measured rather than theoretical, because the KRR is finally trained on real trajectories.
+- Phase 1 is entirely additive: no `model-router.ts` dispatch logic changes, no top-level command surface change, no IPFS registry change. Backward-compatible MINOR bump.
+- Three integration channels match MetaHarness's three contributions β€” analysis, routing, hosts β€” without forcing the kernel's full surface into a position where its 0.x stability would block ruflo releases.
+
+### Negative / risks
+
+- **API stability**: both `metaharness` and `@metaharness/router` are 0.x and ship rapid patch releases. A breaking change in `@metaharness/[email protected]` would require immediate `neural-router.ts` updates. Mitigation: pin to `~0.3.2` in `optionalDependencies`; add `scripts/check-metaharness-compat.mjs` to CI exercising the `Router` constructor with a trivial example to catch runtime breakage before publishing.
+- **Bus factor**: same maintainer as ruflo, MetaHarness, and ruvector. No change from today, but the dependency edge is now explicit.
+- **Sandboxing**: `harness from-repo <url>` clones arbitrary Git URLs. Phase-1 skills NEVER expose this to Claude Code; only `analyze`/`score`/`genome` (pure reads) and `harness-mint` (writes to user-specified target dir, never project root).
+- **GCP dependency**: `harness validate` uses GCP Secret Manager via `gcloud`. Ruflo CI must skip those subcommands (or mock them) β€” explicit `--skip-gcp` flag from the `harness validate` command surface handles this.
+- **Phase-1 MCP plugin spawns subprocesses**: subprocess crashes, timeouts, and stdout-parsing edge cases are now in ruflo's failure surface. Mitigation: hard timeout (60s) per invocation, captured stderr in error responses, structured-JSON output enforced via `--json` flag everywhere.
+
+### Neutral / accepted trade-offs
+
+- Subprocess invocation in Phase 1 (rather than library import) adds ~200ms cold-start overhead per call vs. an embedded library. Acceptable for MCP tools that are not in the hot path; the router path (already library-imported) remains as-is.
+- Maintaining a `ruflo-metaharness` plugin doubles documentation surface for two sibling tools. Mitigation: skill descriptions explicitly point to the upstream MetaHarness docs as canonical for the underlying functionality; the plugin only documents the ruflo-side adaptation.
+
+## Alternatives Considered
+
+**Alternative A: Ignore MetaHarness, build all scaffolding/score/genome natively in ruflo.**
+Rejected. `buildRepoScorecard()`, `buildGenomeReport()`, `scanMcp()`, `buildThreatModel()` are already-tested implementations exposing clean TypeScript APIs. Reimplementing them in ruflo is pure duplication cost with no advantage. The eject path's `rewriteContent()` with attribution-block preservation is subtle.
+
+**Alternative B: Use MetaHarness only as a CLI subprocess everywhere, never as a library import.**
+Partially adopted (this is the Phase-1 plugin posture). Wrong for `@metaharness/router` β€” sub-ms routing latency demands a library import, which ADR-148/149 already accepted.
+
+**Alternative C: Promote `@metaharness/router` from `optionalDependency` to `dependency`.**
+Rejected for now. The triple gate (`CLAUDE_FLOW_ROUTER_NEURAL=1` + artifact + import success) is the right posture until the API stabilizes at 1.0.
+
+**Alternative D: Wait for MetaHarness 1.0 before any further integration beyond ADR-148/149.**
+Rejected. The static-analysis surface (`score`, `genome`, `mcp scan`, `threat-model`) is already mature (475 files, well-typed, pure reads). Waiting creates a window where users discover MetaHarness independently and are confused about its relationship to ruflo. The Phase-1 plugin answers that question without incurring API-stability risk because the integration is via CLI subprocess, not library import.
+
+**Alternative E: Wire `@metaharness/kernel`'s `ToolDispatcher` as the primary MCP dispatch in Phase 1.**
+Rejected. Touching the MCP dispatch core affects all 314 tools and is too high-blast-radius for an early-stage (v0.1.0) component. Deferred to a Phase-3 ADR after the kernel ships a 1.0 with API-stability commitments.
+
+## Open Questions
+
+- Should the Phase-1 plugin's `harness-mint` skill require explicit user confirmation in the Claude Code UI before writing any files? Lean yes β€” destructive-action-confirmation matches ruflo's "executing actions with care" principle.
+- Should the seed-corpus retraining cadence be ad-hoc (Phase-1) or scheduled (e.g., monthly cron in a Phase-2 follow-up)? Defer to Phase-2 once we see the trajectory volume.
+- Does the `oia-audit` background worker (Phase 2) belong in `ruflo-loop-workers` or in `ruflo-metaharness`? Probably the latter, since the audit output is MetaHarness-specific.
+
+## References
+
+- [Research dossier (gist)](https://gist.github.com/ruvnet/19d166ff9acf368c9da4172d91ac9113) β€” full graded-evidence sourcing.
+- [Tracking issue](https://github.com/ruvnet/ruflo/issues) β€” link inserted once opened; phase checklist.
+- ADR-148 β€” Cost-optimal router lifecycle via `@metaharness/router`.
+- ADR-149 β€” Per-model cost-optimal routing (Pareto framing).
+- ADR-097 β€” Federation budget circuit breaker (cost-spend telemetry pattern reused by metaharness plugin).
+- `[email protected]` on npm: <https://www.npmjs.com/package/metaharness>
+- `@metaharness/[email protected]` on npm: <https://www.npmjs.com/package/@metaharness/router>
+- `@metaharness/[email protected]` on npm: <https://www.npmjs.com/package/@metaharness/kernel>
+- Upstream: <https://github.com/ruvnet/agent-harness-generator>