π The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated
TypeScript agentic-aiagentic-frameworkagentic-workflowagentsai-agentsai-assistantai-skillsautonomous-agentsclaudecodexdsh-pluginharnessmcp-servermulti-agentmulti-agent-systemsnpmskillsswarmswarm-intelligencetypescript
feat(intelligence): hybrid retrieval + outcome signal (ADR-078, 3.10.18)
ruv62df8a184db3
diff --git a/docs/benchmarks/runs/pretrain-from-github-2026-05-30T17-15-24-767Z.json b/docs/benchmarks/runs/pretrain-from-github-2026-05-30T17-15-24-767Z.jsonnew file mode 100644index 0000000000..90f1e67f5a--- /dev/null+++ b/docs/benchmarks/runs/pretrain-from-github-2026-05-30T17-15-24-767Z.json@@ -0,0 +1,63 @@+{+ "runAt": "2026-05-30T17:15:24.767Z",+ "benchmark": "pretrain-from-github",+ "source": "git",+ "config": {+ "COMMITS": 200,+ "ISSUES": 30+ },+ "harvest": {+ "commits": 200,+ "issues": 0,+ "total": 200,+ "harvestMs": 5577.96+ },+ "feed": {+ "trained": 200,+ "failed": 0,+ "avgLatencyMs": 12.38,+ "totalMs": 2475.27,+ "sampleFailures": [],+ "verdictMix": {+ "success": 200,+ "partial": 0,+ "reverted": 0,+ "hotfixed": 0+ }+ },+ "seedNeuralStore": {+ "stored": 200,+ "total": 200,+ "seedMs": 95.03+ },+ "before": {+ "trajectoriesRecorded": 0,+ "patternsLearned": 0,+ "signalsProcessed": 0,+ "neuralPatternCount": 185,+ "memoryBridgeTotal": 0+ },+ "after": {+ "trajectoriesRecorded": 385,+ "patternsLearned": 365,+ "signalsProcessed": 0,+ "neuralPatternCount": 385,+ "memoryBridgeTotal": 0+ },+ "deltas": {+ "trajectoriesRecorded": 385,+ "patternsLearned": 365,+ "signalsProcessed": 0,+ "neuralPatternCount": 200,+ "memoryBridgeTotal": 0+ },+ "consistency": {+ "sonaTracksGlobal": false,+ "sonaTracksGlobalDelta": -385,+ "notes": [+ "sona.trajectoriesTotal (0) drifts from globalStats.trajectoriesRecorded (385) by -385 β expected to track within Β±1",+ "memory-bridge unreachable β bridge-dependent counters (post-edit/-command persistence, pretrain bundle) will show 0"+ ]+ },+ "passed": true+}\ No newline at end of filediff --git a/docs/benchmarks/runs/pretrain-from-github-latest.json b/docs/benchmarks/runs/pretrain-from-github-latest.jsonindex ce020f136b..90f1e67f5a 100644--- a/docs/benchmarks/runs/pretrain-from-github-latest.json+++ b/docs/benchmarks/runs/pretrain-from-github-latest.json@@ -1,55 +1,61 @@{- "runAt": "2026-05-30T16:51:28.542Z",+ "runAt": "2026-05-30T17:15:24.767Z","benchmark": "pretrain-from-github",- "source": "all",+ "source": "git","config": {- "COMMITS": 50,+ "COMMITS": 200,"ISSUES": 30},"harvest": {- "commits": 50,- "issues": 30,- "total": 80,- "harvestMs": 598.47+ "commits": 200,+ "issues": 0,+ "total": 200,+ "harvestMs": 5577.96},"feed": {- "trained": 80,+ "trained": 200,"failed": 0,- "avgLatencyMs": 3.36,- "totalMs": 268.65,- "sampleFailures": []+ "avgLatencyMs": 12.38,+ "totalMs": 2475.27,+ "sampleFailures": [],+ "verdictMix": {+ "success": 200,+ "partial": 0,+ "reverted": 0,+ "hotfixed": 0+ }},"seedNeuralStore": {- "stored": 80,- "total": 80,- "seedMs": 29.47+ "stored": 200,+ "total": 200,+ "seedMs": 95.03},"before": {"trajectoriesRecorded": 0,"patternsLearned": 0,"signalsProcessed": 0,- "neuralPatternCount": 15,+ "neuralPatternCount": 185,"memoryBridgeTotal": 0},"after": {- "trajectoriesRecorded": 95,- "patternsLearned": 85,+ "trajectoriesRecorded": 385,+ "patternsLearned": 365,"signalsProcessed": 0,- "neuralPatternCount": 95,+ "neuralPatternCount": 385,"memoryBridgeTotal": 0},"deltas": {- "trajectoriesRecorded": 95,- "patternsLearned": 85,+ "trajectoriesRecorded": 385,+ "patternsLearned": 365,"signalsProcessed": 0,- "neuralPatternCount": 80,+ "neuralPatternCount": 200,"memoryBridgeTotal": 0},"consistency": {"sonaTracksGlobal": false,- "sonaTracksGlobalDelta": -95,+ "sonaTracksGlobalDelta": -385,"notes": [- "sona.trajectoriesTotal (0) drifts from globalStats.trajectoriesRecorded (95) by -95 β expected to track within Β±1",+ "sona.trajectoriesTotal (0) drifts from globalStats.trajectoriesRecorded (385) by -385 β expected to track within Β±1","memory-bridge unreachable β bridge-dependent counters (post-edit/-command persistence, pretrain bundle) will show 0"]},diff --git a/docs/benchmarks/runs/pretrained-retrieval-2026-05-30T17-18-50-632Z.json b/docs/benchmarks/runs/pretrained-retrieval-2026-05-30T17-18-50-632Z.jsonnew file mode 100644index 0000000000..9351de2de1--- /dev/null+++ b/docs/benchmarks/runs/pretrained-retrieval-2026-05-30T17-18-50-632Z.json@@ -0,0 +1,559 @@+{+ "runAt": "2026-05-30T17:18:50.632Z",+ "benchmark": "pretrained-retrieval",+ "mode": "hybrid",+ "storeSize": 385,+ "queries": 10,+ "matchedQueries": 10,+ "matchRate": 1,+ "top1HitRate": 0.5,+ "top3HitRate": 0.7,+ "mrr3": 0.5833,+ "top1Diversity": 0.8,+ "top3DupRate": 0,+ "avgQueryLatencyMs": 40.6,+ "totalQueryMs": 405.98,+ "results": [+ {+ "query": "how was the Opus model alias fixed",+ "matched": true,+ "top1Relevant": false,+ "top3Relevant": true,+ "firstRelevantRank": 2,+ "topK": [+ {+ "id": "pattern-1780161237205-9-48p8",+ "name": "chore(release): bump 3.10.10 β 3.10.11 (4-issue bug cluster)",+ "type": "history-commit",+ "score": 0.8613360283563845,+ "cosineScore": 0.11877348981913242,+ "bm25Score": 14.747970179516324,+ "mmrScore": 0.4306680141781922+ },+ {+ "id": "pattern-1780159888525-1o-qiri",+ "name": "Is Ruflo safe on Claude Opus 4.8? β `opus` alias still pins to 4.7, and literal model ids silently d",+ "type": "history-issue",+ "score": 0.6843424344550606,+ "cosineScore": 0.003778643529534665,+ "bm25Score": 18.37549252404469,+ "mmrScore": 0.3421712172275303+ },+ {+ "id": "pattern-1780161324702-4r-00fd",+ "name": "feat(graph-intelligence): #2044 ADR-123 Phase 1 β ruflo-graph-intelligence plugin foundation",+ "type": "history-commit",+ "score": 0.6,+ "cosineScore": 0.14559488017897365,+ "bm25Score": 0,+ "mmrScore": 0.2867919386912716+ },+ {+ "id": "pattern-1780161324689-23-qwbk",+ "name": "fix(mcp): #2042 β agent_execute routes through v3 provider system (Anthropic / OpenRouter / Ollama)",+ "type": "history-commit",+ "score": 0.5804934595199714,+ "cosineScore": 0.1117424282926092,+ "bm25Score": 2.5653547216178456,+ "mmrScore": 0.28055883354168704+ },+ {+ "id": "pattern-1780161237206-a-bqc4",+ "name": "fix(intelligence): 4-issue bug cluster β Opus 4.8 + uptime + MCP orphan + Q-encoder",+ "type": "history-commit",+ "score": 0.5002402589094885,+ "cosineScore": 0.0020688282745718813,+ "bm25Score": 10.09290314376738,+ "mmrScore": 0.25012012945474427+ }+ ]+ },+ {+ "query": "self-learning wiring task-completed pretrain",+ "matched": true,+ "top1Relevant": true,+ "top3Relevant": true,+ "firstRelevantRank": 1,+ "topK": [+ {+ "id": "pattern-1780158992310-a-1p27",+ "name": "Self-learning reports success but persists nothing (3.10.6): signalsProcessed dead, task-completed s",+ "type": "history-issue",+ "score": 0.7749415691985588,+ "cosineScore": 0.04246140228195686,+ "bm25Score": 18.052348783755328,+ "mmrScore": 0.3874707845992794+ },+ {+ "id": "pattern-1780158992307-3-p1z2",+ "name": "chore(release): bump 3.10.13 β 3.10.14 (ADR-074 self-learning wiring)",+ "type": "history-commit",+ "score": 0.7013997452949954,+ "cosineScore": 0.09545006956277129,+ "bm25Score": 9.89227235345964,+ "mmrScore": 0.3506998726474977+ },+ {+ "id": "pattern-1780161324689-25-5hh2",+ "name": "chore(release): publish 3.7.0-alpha.77 β #2098 #2093 #2085 fixes",+ "type": "history-commit",+ "score": 0.6,+ "cosineScore": 0.15363730484346572,+ "bm25Score": 0,+ "mmrScore": 0.3+ },+ {+ "id": "pattern-1780161237204-3-zs1a",+ "name": "Merge fix/self-learning-2245: ADR-074 self-learning wiring (closes #2245)",+ "type": "history-commit",+ "score": 0.5918785037100176,+ "cosineScore": 0.028229282902256594,+ "bm25Score": 11.09080619601972,+ "mmrScore": 0.2731403286839812+ },+ {+ "id": "pattern-1780161324694-3b-h7wi",+ "name": "Merge pull request #2071 from ruvnet/feat/adr-126-phase-4-5-trust-pipeline",+ "type": "history-commit",+ "score": 0.5497170195764974,+ "cosineScore": 0.12879817434673874,+ "bm25Score": 0,+ "mmrScore": 0.252278816949607+ }+ ]+ },+ {+ "query": "deterministic codemod engine var-to-const",+ "matched": true,+ "top1Relevant": true,+ "top3Relevant": true,+ "firstRelevantRank": 1,+ "topK": [+ {+ "id": "pattern-1780161237206-c-jffq",+ "name": "Merge feat/deterministic-tier1-codemods: deterministic Tier-1 codemods (ADR-143)",+ "type": "history-commit",+ "score": 0.7563913742689558,+ "cosineScore": 0.040446535166730725,+ "bm25Score": 8.478971524833781,+ "mmrScore": 0.3781956871344779+ },+ {+ "id": "pattern-1780159888520-s-5b5g",+ "name": "feat(gaia): #2156 iter 62 β convergence layer (deterministic finalization)",+ "type": "history-commit",+ "score": 0.7124892446299289,+ "cosineScore": 0.08054377890438601,+ "bm25Score": 5.049873510910536,+ "mmrScore": 0.35072498046733647+ },+ {+ "id": "pattern-1780161237206-b-hgnz",+ "name": "feat(cost-tracker): wire codemod benchmark into cost-trend (ADR-143 follow-up)",+ "type": "history-commit",+ "score": 0.6666354734387321,+ "cosineScore": 0.008640664241102285,+ "bm25Score": 8.032816557341778,+ "mmrScore": 0.33331773671936604+ },+ {+ "id": "pattern-1780161324695-3k-3187",+ "name": "Merge pull request #2067 from ruvnet/docs/trim-redundant-badges",+ "type": "history-commit",+ "score": 0.6,+ "cosineScore": 0.11852022008519922,+ "bm25Score": 0,+ "mmrScore": 0.29966268778718885+ },+ {+ "id": "pattern-1780161324698-3u-c1g1",+ "name": "chore(release): publish 3.7.0-alpha.71 + @claude-flow/[email protected]",+ "type": "history-commit",+ "score": 0.5970925328149753,+ "cosineScore": 0.11723942732447756,+ "bm25Score": 0,+ "mmrScore": 0.2894676570508941+ }+ ]+ },+ {+ "query": "MCP server orphan leak parent-death",+ "matched": true,+ "top1Relevant": false,+ "top3Relevant": true,+ "firstRelevantRank": 3,+ "topK": [+ {+ "id": "pattern-1780161237205-9-48p8",+ "name": "chore(release): bump 3.10.10 β 3.10.11 (4-issue bug cluster)",+ "type": "history-commit",+ "score": 0.7581882286396684,+ "cosineScore": 0.017234812587240394,+ "bm25Score": 20.48711361088179,+ "mmrScore": 0.3790941143198342+ },+ {+ "id": "pattern-1780161324703-54-114j",+ "name": "feat(browser): #2041 ADR-122 Phase 5 β cost-aware routing + GOAP preflight",+ "type": "history-commit",+ "score": 0.5946483634011628,+ "cosineScore": 0.12641500425453153,+ "bm25Score": 0,+ "mmrScore": 0.2973241817005814+ },+ {+ "id": "pattern-1780161237212-1n-sokv",+ "name": "MCP servers orphan on every Claude Code restart β ~1 GB leaked, and a stale server silently serves t",+ "type": "history-issue",+ "score": 0.6188847204623305,+ "cosineScore": -0.005021665326373356,+ "bm25Score": 15.821127280743537,+ "mmrScore": 0.2826895519430563+ },+ {+ "id": "pattern-1780159888524-18-ks7x",+ "name": "feat(ci): #2132 β cross-platform audit for plugin hooks.json commands",+ "type": "history-commit",+ "score": 0.5918790559890619,+ "cosineScore": 0.12513633834180263,+ "bm25Score": 0,+ "mmrScore": 0.2757305090107404+ },+ {+ "id": "pattern-1780161324688-1w-n21p",+ "name": "perf(guidance): M4 quantization β 1-bit RaBitQ-style signatures, 2.7x at N=1000",+ "type": "history-commit",+ "score": 0.6,+ "cosineScore": 0.128886003056017,+ "bm25Score": 0,+ "mmrScore": 0.25419649306537573+ }+ ]+ },+ {+ "query": "unified learning stats aggregator",+ "matched": true,+ "top1Relevant": true,+ "top3Relevant": true,+ "firstRelevantRank": 1,+ "topK": [+ {+ "id": "pattern-1780160010139-1-tp0d",+ "name": "feat(memory): unified learning-stats aggregator (ADR-075)",+ "type": "history-commit",+ "score": 0.6535228171782452,+ "cosineScore": -0.03179548242068774,+ "bm25Score": 13.229926107641308,+ "mmrScore": 0.3267614085891226+ },+ {+ "id": "pattern-1780158992306-1-jwma",+ "name": "feat(intelligence): unified learning-stats aggregator (ADR-075, #2245 follow-up)",+ "type": "history-commit",+ "score": 0.5893416797764868,+ "cosineScore": -0.06202219389140224,+ "bm25Score": 12.946176756248445,+ "mmrScore": 0.2946708398882434+ },+ {+ "id": "pattern-1780159888526-1v-6kuu",+ "name": "bug: flashAttention reports contradictory state across MCP tools (system_info=false vs hooks_intelli",+ "type": "history-issue",+ "score": 0.6,+ "cosineScore": 0.15655822549223372,+ "bm25Score": 0,+ "mmrScore": 0.2757020128669746+ },+ {+ "id": "pattern-1780161237203-0-z767",+ "name": "feat(intelligence): pretrain self-learning from repo history (ADR-077, 3.10.17)",+ "type": "history-commit",+ "score": 0.5420344834308211,+ "cosineScore": 0.08157768176522591,+ "bm25Score": 2.6447087795290756,+ "mmrScore": 0.27101724171541053+ },+ {+ "id": "pattern-1780159888520-q-3lmr",+ "name": "fix(init): #2206 #2207 #2208 β wizard server key, init detector, CLAUDE.md backup (#2214)",+ "type": "history-commit",+ "score": 0.5626313450839108,+ "cosineScore": 0.13624368513537335,+ "bm25Score": 0,+ "mmrScore": 0.2658356829430097+ }+ ]+ },+ {+ "query": "structured distillation 4-field schema",+ "matched": true,+ "top1Relevant": true,+ "top3Relevant": true,+ "firstRelevantRank": 1,+ "topK": [+ {+ "id": "pattern-1780161221198-2-kxx1",+ "name": "feat(intelligence): structured distillation (ADR-076)",+ "type": "history-commit",+ "score": 0.7436629268946584,+ "cosineScore": 0.01303151951388954,+ "bm25Score": 23.617385640561277,+ "mmrScore": 0.3718314634473292+ },+ {+ "id": "pattern-1780159888518-b-vf8y",+ "name": "Merge feat/deterministic-tier1-codemods: deterministic Tier-1 codemods (ADR-143)",+ "type": "history-commit",+ "score": 0.5899985175159177,+ "cosineScore": 0.11334191563527016,+ "bm25Score": 0,+ "mmrScore": 0.29499925875795885+ },+ {+ "id": "pattern-1780161324689-22-p22q",+ "name": "Merge pull request #2101 from ruvnet/fix/2078-claudemd-coauthor-rule",+ "type": "history-commit",+ "score": 0.6,+ "cosineScore": 0.11741462257643585,+ "bm25Score": 0,+ "mmrScore": 0.27832122285750827+ },+ {+ "id": "pattern-1780160010138-0-cefy",+ "name": "fix(intelligence): wire hooks_task-completed to recordTrajectory",+ "type": "history-commit",+ "score": 0.5450536587835099,+ "cosineScore": 0.09503990506628572,+ "bm25Score": 0,+ "mmrScore": 0.27101959503190853+ },+ {+ "id": "pattern-1780161324698-3w-5c7h",+ "name": "feat(memory): #2061 ADR-125 Phase 5 β graceful retrieval degradation + FTS5 keyword fallback",+ "type": "history-commit",+ "score": 0.5750560127842083,+ "cosineScore": 0.1072571734146955,+ "bm25Score": 0,+ "mmrScore": 0.2557069865449775+ }+ ]+ },+ {+ "query": "SQL injection migrate.ts table identifier",+ "matched": true,+ "top1Relevant": false,+ "top3Relevant": false,+ "firstRelevantRank": -1,+ "topK": [+ {+ "id": "pattern-1780161324703-51-v1b7",+ "name": "docs(browser): #2041 ADR-122 security audit + substrate benchmarks + announcement draft",+ "type": "history-commit",+ "score": 0.6510393622687756,+ "cosineScore": 0.019588773519894014,+ "bm25Score": 6.206553480055758,+ "mmrScore": 0.3255196811343878+ },+ {+ "id": "pattern-1780161237211-19-7z58",+ "name": "feat(ci): #2132 β cross-platform audit for plugin hooks.json commands",+ "type": "history-commit",+ "score": 0.6121441415703033,+ "cosineScore": 0.09711664650832315,+ "bm25Score": 2.5401471715239787,+ "mmrScore": 0.3050938890510861+ },+ {+ "id": "pattern-1780161324691-2i-tm1b",+ "name": "Merge pull request #2090 from ruvnet/docs/adr-127-github-stack-modernization",+ "type": "history-commit",+ "score": 0.6,+ "cosineScore": 0.15794464752821977,+ "bm25Score": 0,+ "mmrScore": 0.3+ },+ {+ "id": "pattern-1780161237205-6-7kk3",+ "name": "chore(release): bump 3.10.12 β 3.10.13 + [email protected]",+ "type": "history-commit",+ "score": 0.5272599789668341,+ "cosineScore": -0.06928700588415701,+ "bm25Score": 7.325974016308216,+ "mmrScore": 0.2552980359566925+ },+ {+ "id": "pattern-1780161324689-21-o5x6",+ "name": "fix(smoke): #2042 β make providers smoke static-only (no build dependency)",+ "type": "history-commit",+ "score": 0.5486082159087727,+ "cosineScore": 0.13324219046664132,+ "bm25Score": 0,+ "mmrScore": 0.24417206666200053+ }+ ]+ },+ {+ "query": "recall@k HNSW benchmark harness",+ "matched": true,+ "top1Relevant": true,+ "top3Relevant": true,+ "firstRelevantRank": 1,+ "topK": [+ {+ "id": "pattern-1780161324693-2z-5x4a",+ "name": "feat(neural-trader): #48 β benchmark suite (signal, backtest, memory-recall)",+ "type": "history-commit",+ "score": 0.8018266090177799,+ "cosineScore": 0.08967419136034757,+ "bm25Score": 11.069668624696455,+ "mmrScore": 0.40091330450888996+ },+ {+ "id": "pattern-1780161237205-6-7kk3",+ "name": "chore(release): bump 3.10.12 β 3.10.13 + [email protected]",+ "type": "history-commit",+ "score": 0.6930655013125043,+ "cosineScore": 0.013638755437439873,+ "bm25Score": 12.520917564394331,+ "mmrScore": 0.34653275065625216+ },+ {+ "id": "pattern-1780161324698-3y-v3g2",+ "name": "feat(memory): #2061 ADR-125 Phase 3 β persistent HNSW snapshot + restore",+ "type": "history-commit",+ "score": 0.5894330046400449,+ "cosineScore": 0.08630266804079345,+ "bm25Score": 4.636571427447146,+ "mmrScore": 0.2822710787648221+ },+ {+ "id": "pattern-1780159888524-1d-x3m6",+ "name": "fix(bench): ADR-130 P6 β CI-friendly benchmark using single-session inserts",+ "type": "history-commit",+ "score": 0.5338765470062481,+ "cosineScore": 0.06474451013559841,+ "bm25Score": 4.274258657174938,+ "mmrScore": 0.26693827350312405+ },+ {+ "id": "pattern-1780158992316-e-0arj",+ "name": "SHABHNAM JAYAV 8Γ·58/π Rollback Incident: [Brief Description] not work",+ "type": "history-issue",+ "score": 0.6,+ "cosineScore": 0.16408597847506598,+ "bm25Score": 0,+ "mmrScore": 0.2563579067611487+ }+ ]+ },+ {+ "query": "Q-learning encoder keyword block",+ "matched": true,+ "top1Relevant": false,+ "top3Relevant": false,+ "firstRelevantRank": -1,+ "topK": [+ {+ "id": "pattern-1780161237205-9-48p8",+ "name": "chore(release): bump 3.10.10 β 3.10.11 (4-issue bug cluster)",+ "type": "history-commit",+ "score": 0.710559958201223,+ "cosineScore": 0.029870791404767158,+ "bm25Score": 13.600488646605163,+ "mmrScore": 0.3552799791006115+ },+ {+ "id": "pattern-1780161324698-3u-c1g1",+ "name": "chore(release): publish 3.7.0-alpha.71 + @claude-flow/[email protected]",+ "type": "history-commit",+ "score": 0.6325338902596626,+ "cosineScore": 0.10356354291635028,+ "bm25Score": 3.099284643164071,+ "mmrScore": 0.30228090848432076+ },+ {+ "id": "pattern-1780161324688-1z-fu6u",+ "name": "chore(release): publish 3.7.0-alpha.78 β #2042 #2078 fixes",+ "type": "history-commit",+ "score": 0.6,+ "cosineScore": 0.12079199972660587,+ "bm25Score": 0,+ "mmrScore": 0.29654148767587263+ },+ {+ "id": "pattern-1780161237213-1s-xof1",+ "name": "Statusline shows `RuFlo V3.6` for global npm installs β statusline-generator version probe never che",+ "type": "history-issue",+ "score": 0.61629844815063,+ "cosineScore": 0.10389287629109815,+ "bm25Score": 2.336254643968555,+ "mmrScore": 0.2783942537276099+ },+ {+ "id": "pattern-1780159888519-m-r8lt",+ "name": "docs(adr): ADR-129 status β Accepted + Gap 2 implementation record (#2201)",+ "type": "history-commit",+ "score": 0.56928734499748,+ "cosineScore": 0.10618479339045693,+ "bm25Score": 0,+ "mmrScore": 0.27450268662114563+ }+ ]+ },+ {+ "query": "security hardening crypto random IDs",+ "matched": true,+ "top1Relevant": false,+ "top3Relevant": false,+ "firstRelevantRank": -1,+ "topK": [+ {+ "id": "pattern-1780161237205-6-7kk3",+ "name": "chore(release): bump 3.10.12 β 3.10.13 + [email protected]",+ "type": "history-commit",+ "score": 0.7292296778666876,+ "cosineScore": 0.03253431220092657,+ "bm25Score": 10.582768112698659,+ "mmrScore": 0.3646148389333438+ },+ {+ "id": "pattern-1780161324704-5d-ssyg",+ "name": "fix(hooks): #2017 β defensive pre-bash + new CI gate against silent-swallow shape",+ "type": "history-commit",+ "score": 0.6464283623857547,+ "cosineScore": 0.11760821356909369,+ "bm25Score": 2.8379359683329337,+ "mmrScore": 0.30529026970129103+ },+ {+ "id": "pattern-1780161324699-43-nlue",+ "name": "feat(memory): #2061 ADR-125 Phase 1 β MemoryService canonical entry point + deprecate UnifiedMemoryS",+ "type": "history-commit",+ "score": 0.6,+ "cosineScore": 0.14226244116789438,+ "bm25Score": 0,+ "mmrScore": 0.2970161868092974+ },+ {+ "id": "pattern-1780161324700-4b-ieca",+ "name": "Merge pull request #2050 from ruvnet/feat/ci-supply-chain-hardening",+ "type": "history-commit",+ "score": 0.592484117643453,+ "cosineScore": 0.052350663373743465,+ "bm25Score": 5.6711645256634045,+ "mmrScore": 0.28617992493599675+ },+ {+ "id": "pattern-1780161324693-33-7swi",+ "name": "fix(init): #2078 β use no-reply bot email for opt-in Co-Authored-By",+ "type": "history-commit",+ "score": 0.5500381877424747,+ "cosineScore": 0.12201570165957405,+ "bm25Score": 0,+ "mmrScore": 0.2532795338785324+ }+ ]+ }+ ],+ "passed": true+}\ No newline at end of filediff --git a/docs/benchmarks/runs/pretrained-retrieval-latest.json b/docs/benchmarks/runs/pretrained-retrieval-latest.jsonindex 1fb3622fd5..9351de2de1 100644--- a/docs/benchmarks/runs/pretrained-retrieval-latest.json+++ b/docs/benchmarks/runs/pretrained-retrieval-latest.json@@ -1,370 +1,556 @@{- "runAt": "2026-05-30T16:51:47.770Z",+ "runAt": "2026-05-30T17:18:50.632Z","benchmark": "pretrained-retrieval",- "storeSize": 95,+ "mode": "hybrid",+ "storeSize": 385,"queries": 10,"matchedQueries": 10,"matchRate": 1,- "avgQueryLatencyMs": 7.67,- "totalQueryMs": 76.74,+ "top1HitRate": 0.5,+ "top3HitRate": 0.7,+ "mrr3": 0.5833,+ "top1Diversity": 0.8,+ "top3DupRate": 0,+ "avgQueryLatencyMs": 40.6,+ "totalQueryMs": 405.98,"results": [{"query": "how was the Opus model alias fixed","matched": true,+ "top1Relevant": false,+ "top3Relevant": true,+ "firstRelevantRank": 2,"topK": [{- "id": "pattern-1780158992309-8-px2n",+ "id": "pattern-1780161237205-9-48p8","name": "chore(release): bump 3.10.10 β 3.10.11 (4-issue bug cluster)","type": "history-commit",- "score": 0.11877348981913242+ "score": 0.8613360283563845,+ "cosineScore": 0.11877348981913242,+ "bm25Score": 14.747970179516324,+ "mmrScore": 0.4306680141781922},{- "id": "pattern-1780159888517-8-e2lg",- "name": "chore(release): bump 3.10.10 β 3.10.11 (4-issue bug cluster)",- "type": "history-commit",- "score": 0.11877348981913242+ "id": "pattern-1780159888525-1o-qiri",+ "name": "Is Ruflo safe on Claude Opus 4.8? β `opus` alias still pins to 4.7, and literal model ids silently d",+ "type": "history-issue",+ "score": 0.6843424344550606,+ "cosineScore": 0.003778643529534665,+ "bm25Score": 18.37549252404469,+ "mmrScore": 0.3421712172275303},{- "id": "pattern-1780159888525-1n-2zyq",- "name": "MCP servers orphan on every Claude Code restart β ~1 GB leaked, and a stale server silently serves t",- "type": "history-issue",- "score": 0.10838591586557422+ "id": "pattern-1780161324702-4r-00fd",+ "name": "feat(graph-intelligence): #2044 ADR-123 Phase 1 β ruflo-graph-intelligence plugin foundation",+ "type": "history-commit",+ "score": 0.6,+ "cosineScore": 0.14559488017897365,+ "bm25Score": 0,+ "mmrScore": 0.2867919386912716},{- "id": "pattern-1780158992308-4-a45i",- "name": "feat(intelligence): self-learning wiring + honest multi-path output (ADR-074, #2245)",+ "id": "pattern-1780161324689-23-qwbk",+ "name": "fix(mcp): #2042 β agent_execute routes through v3 provider system (Anthropic / OpenRouter / Ollama)","type": "history-commit",- "score": 0.09638388002926226+ "score": 0.5804934595199714,+ "cosineScore": 0.1117424282926092,+ "bm25Score": 2.5653547216178456,+ "mmrScore": 0.28055883354168704},{- "id": "pattern-1780159888516-4-lo8t",- "name": "feat(intelligence): self-learning wiring + honest multi-path output (ADR-074, #2245)",+ "id": "pattern-1780161237206-a-bqc4",+ "name": "fix(intelligence): 4-issue bug cluster β Opus 4.8 + uptime + MCP orphan + Q-encoder","type": "history-commit",- "score": 0.09638388002926226+ "score": 0.5002402589094885,+ "cosineScore": 0.0020688282745718813,+ "bm25Score": 10.09290314376738,+ "mmrScore": 0.25012012945474427}]},{"query": "self-learning wiring task-completed pretrain","matched": true,+ "top1Relevant": true,+ "top3Relevant": true,+ "firstRelevantRank": 1,"topK": [{- "id": "pattern-1780159888526-1t-sjm0",- "name": "agentdb pins better-sqlite3@^11.8.1, which has no Node 24/26 prebuilt and fails to compile β silent ",+ "id": "pattern-1780158992310-a-1p27",+ "name": "Self-learning reports success but persists nothing (3.10.6): signalsProcessed dead, task-completed s","type": "history-issue",- "score": 0.11529152006391366+ "score": 0.7749415691985588,+ "cosineScore": 0.04246140228195686,+ "bm25Score": 18.052348783755328,+ "mmrScore": 0.3874707845992794},{- "id": "pattern-1780159888521-y-mk0f",- "name": "fix(statusline): #2195 β generator delegates to hooks-statusline --json; fix ADR count, vectors, int",+ "id": "pattern-1780158992307-3-p1z2",+ "name": "chore(release): bump 3.10.13 β 3.10.14 (ADR-074 self-learning wiring)","type": "history-commit",- "score": 0.11154879452211397+ "score": 0.7013997452949954,+ "cosineScore": 0.09545006956277129,+ "bm25Score": 9.89227235345964,+ "mmrScore": 0.3506998726474977},{- "id": "pattern-1780158992307-3-p1z2",- "name": "chore(release): bump 3.10.13 β 3.10.14 (ADR-074 self-learning wiring)",+ "id": "pattern-1780161324689-25-5hh2",+ "name": "chore(release): publish 3.7.0-alpha.77 β #2098 #2093 #2085 fixes","type": "history-commit",- "score": 0.09545006956277129+ "score": 0.6,+ "cosineScore": 0.15363730484346572,+ "bm25Score": 0,+ "mmrScore": 0.3},{- "id": "pattern-1780159888516-3-4vmx",- "name": "chore(release): bump 3.10.13 β 3.10.14 (ADR-074 self-learning wiring)",+ "id": "pattern-1780161237204-3-zs1a",+ "name": "Merge fix/self-learning-2245: ADR-074 self-learning wiring (closes #2245)","type": "history-commit",- "score": 0.09545006956277129+ "score": 0.5918785037100176,+ "cosineScore": 0.028229282902256594,+ "bm25Score": 11.09080619601972,+ "mmrScore": 0.2731403286839812},{- "id": "pattern-1780159888522-15-d9b0",- "name": "chore: project-wide quality sweep β no dead code, no slop, witness-validated (#2139)",+ "id": "pattern-1780161324694-3b-h7wi",+ "name": "Merge pull request #2071 from ruvnet/feat/adr-126-phase-4-5-trust-pipeline","type": "history-commit",- "score": 0.08963554948414637+ "score": 0.5497170195764974,+ "cosineScore": 0.12879817434673874,+ "bm25Score": 0,+ "mmrScore": 0.252278816949607}]},{"query": "deterministic codemod engine var-to-const","matched": true,+ "top1Relevant": true,+ "top3Relevant": true,+ "firstRelevantRank": 1,"topK": [{- "id": "pattern-1780159888526-1w-jptg",- "name": "wizard: unconditional CLAUDE.md overwrite destroys project guidance",- "type": "history-issue",- "score": 0.10896682436648679+ "id": "pattern-1780161237206-c-jffq",+ "name": "Merge feat/deterministic-tier1-codemods: deterministic Tier-1 codemods (ADR-143)",+ "type": "history-commit",+ "score": 0.7563913742689558,+ "cosineScore": 0.040446535166730725,+ "bm25Score": 8.478971524833781,+ "mmrScore": 0.3781956871344779},{- "id": "pattern-1780159888520-q-3lmr",- "name": "fix(init): #2206 #2207 #2208 β wizard server key, init detector, CLAUDE.md backup (#2214)",+ "id": "pattern-1780159888520-s-5b5g",+ "name": "feat(gaia): #2156 iter 62 β convergence layer (deterministic finalization)","type": "history-commit",- "score": 0.09952345049620362+ "score": 0.7124892446299289,+ "cosineScore": 0.08054377890438601,+ "bm25Score": 5.049873510910536,+ "mmrScore": 0.35072498046733647},{- "id": "pattern-1780159888522-11-v1dd",- "name": "fix(release): #2151 β three-way version lockstep + CI guard (#2153)",+ "id": "pattern-1780161237206-b-hgnz",+ "name": "feat(cost-tracker): wire codemod benchmark into cost-trend (ADR-143 follow-up)","type": "history-commit",- "score": 0.09582419172696183+ "score": 0.6666354734387321,+ "cosineScore": 0.008640664241102285,+ "bm25Score": 8.032816557341778,+ "mmrScore": 0.33331773671936604},{- "id": "pattern-1780159888525-1q-f8lc",- "name": "[Dream Cycle 2026-05-29] swarm: SWARM+ hierarchical PBFT 97-98% latency gain at 1K agents + ruview-i",- "type": "history-issue",- "score": 0.09226441405704325+ "id": "pattern-1780161324695-3k-3187",+ "name": "Merge pull request #2067 from ruvnet/docs/trim-redundant-badges",+ "type": "history-commit",+ "score": 0.6,+ "cosineScore": 0.11852022008519922,+ "bm25Score": 0,+ "mmrScore": 0.29966268778718885},{- "id": "pattern-1780159888526-1u-8o2f",- "name": "bug(statusline): patterns counter misreads encrypted RFE1 files as SQLite (shows ~2.47B garbage)",- "type": "history-issue",- "score": 0.09193349901930063+ "id": "pattern-1780161324698-3u-c1g1",+ "name": "chore(release): publish 3.7.0-alpha.71 + @claude-flow/[email protected]",+ "type": "history-commit",+ "score": 0.5970925328149753,+ "cosineScore": 0.11723942732447756,+ "bm25Score": 0,+ "mmrScore": 0.2894676570508941}]},{"query": "MCP server orphan leak parent-death","matched": true,+ "top1Relevant": false,+ "top3Relevant": true,+ "firstRelevantRank": 3,"topK": [{- "id": "pattern-1780159888524-18-ks7x",- "name": "feat(ci): #2132 β cross-platform audit for plugin hooks.json commands",+ "id": "pattern-1780161237205-9-48p8",+ "name": "chore(release): bump 3.10.10 β 3.10.11 (4-issue bug cluster)","type": "history-commit",- "score": 0.12513633834180263+ "score": 0.7581882286396684,+ "cosineScore": 0.017234812587240394,+ "bm25Score": 20.48711361088179,+ "mmrScore": 0.3790941143198342},{- "id": "pattern-1780158992308-6-4r0m",- "name": "chore: remove conflicting root overrides.agentdb (kept in ruflo wrapper)",+ "id": "pattern-1780161324703-54-114j",+ "name": "feat(browser): #2041 ADR-122 Phase 5 β cost-aware routing + GOAP preflight","type": "history-commit",- "score": 0.10135302749253655+ "score": 0.5946483634011628,+ "cosineScore": 0.12641500425453153,+ "bm25Score": 0,+ "mmrScore": 0.2973241817005814},{- "id": "pattern-1780159888517-6-tsjt",- "name": "chore: remove conflicting root overrides.agentdb (kept in ruflo wrapper)",- "type": "history-commit",- "score": 0.10135302749253655+ "id": "pattern-1780161237212-1n-sokv",+ "name": "MCP servers orphan on every Claude Code restart β ~1 GB leaked, and a stale server silently serves t",+ "type": "history-issue",+ "score": 0.6188847204623305,+ "cosineScore": -0.005021665326373356,+ "bm25Score": 15.821127280743537,+ "mmrScore": 0.2826895519430563},{- "id": "pattern-1780159888523-16-kv60",- "name": "chore(deps): bump pnpm/action-setup from 2 to 6",+ "id": "pattern-1780159888524-18-ks7x",+ "name": "feat(ci): #2132 β cross-platform audit for plugin hooks.json commands","type": "history-commit",- "score": 0.09445735229713169+ "score": 0.5918790559890619,+ "cosineScore": 0.12513633834180263,+ "bm25Score": 0,+ "mmrScore": 0.2757305090107404},{- "id": "pattern-1780159888518-b-vf8y",- "name": "Merge feat/deterministic-tier1-codemods: deterministic Tier-1 codemods (ADR-143)",+ "id": "pattern-1780161324688-1w-n21p",+ "name": "perf(guidance): M4 quantization β 1-bit RaBitQ-style signatures, 2.7x at N=1000","type": "history-commit",- "score": 0.0832140869143177+ "score": 0.6,+ "cosineScore": 0.128886003056017,+ "bm25Score": 0,+ "mmrScore": 0.25419649306537573}]},{"query": "unified learning stats aggregator","matched": true,+ "top1Relevant": true,+ "top3Relevant": true,+ "firstRelevantRank": 1,"topK": [{- "id": "pattern-1780159888526-1v-6kuu",- "name": "bug: flashAttention reports contradictory state across MCP tools (system_info=false vs hooks_intelli",- "type": "history-issue",- "score": 0.15655822549223372+ "id": "pattern-1780160010139-1-tp0d",+ "name": "feat(memory): unified learning-stats aggregator (ADR-075)",+ "type": "history-commit",+ "score": 0.6535228171782452,+ "cosineScore": -0.03179548242068774,+ "bm25Score": 13.229926107641308,+ "mmrScore": 0.3267614085891226},{- "id": "pattern-1780159888520-q-3lmr",- "name": "fix(init): #2206 #2207 #2208 β wizard server key, init detector, CLAUDE.md backup (#2214)",+ "id": "pattern-1780158992306-1-jwma",+ "name": "feat(intelligence): unified learning-stats aggregator (ADR-075, #2245 follow-up)","type": "history-commit",- "score": 0.13624368513537335+ "score": 0.5893416797764868,+ "cosineScore": -0.06202219389140224,+ "bm25Score": 12.946176756248445,+ "mmrScore": 0.2946708398882434},{- "id": "pattern-1780159888526-21-8ofz",- "name": "[Dream Cycle 2026-05-27] intelligence: SRΒ²AM 8B=120β355B via simulative planning β 95% token gap + c",+ "id": "pattern-1780159888526-1v-6kuu",+ "name": "bug: flashAttention reports contradictory state across MCP tools (system_info=false vs hooks_intelli","type": "history-issue",- "score": 0.1137043910586627+ "score": 0.6,+ "cosineScore": 0.15655822549223372,+ "bm25Score": 0,+ "mmrScore": 0.2757020128669746},{- "id": "pattern-1780159888525-1o-qiri",- "name": "Is Ruflo safe on Claude Opus 4.8? β `opus` alias still pins to 4.7, and literal model ids silently d",- "type": "history-issue",- "score": 0.10722082498011774+ "id": "pattern-1780161237203-0-z767",+ "name": "feat(intelligence): pretrain self-learning from repo history (ADR-077, 3.10.17)",+ "type": "history-commit",+ "score": 0.5420344834308211,+ "cosineScore": 0.08157768176522591,+ "bm25Score": 2.6447087795290756,+ "mmrScore": 0.27101724171541053},{- "id": "pattern-1780159888527-27-s5ye",- "name": "track: project-wide quality sweep (chore/project-wide-quality-sweep)",- "type": "history-issue",- "score": 0.0779117154728549+ "id": "pattern-1780159888520-q-3lmr",+ "name": "fix(init): #2206 #2207 #2208 β wizard server key, init detector, CLAUDE.md backup (#2214)",+ "type": "history-commit",+ "score": 0.5626313450839108,+ "cosineScore": 0.13624368513537335,+ "bm25Score": 0,+ "mmrScore": 0.2658356829430097}]},{"query": "structured distillation 4-field schema","matched": true,+ "top1Relevant": true,+ "top3Relevant": true,+ "firstRelevantRank": 1,"topK": [{- "id": "pattern-1780159888518-b-vf8y",- "name": "Merge feat/deterministic-tier1-codemods: deterministic Tier-1 codemods (ADR-143)",+ "id": "pattern-1780161221198-2-kxx1",+ "name": "feat(intelligence): structured distillation (ADR-076)","type": "history-commit",- "score": 0.11334191563527016+ "score": 0.7436629268946584,+ "cosineScore": 0.01303151951388954,+ "bm25Score": 23.617385640561277,+ "mmrScore": 0.3718314634473292},{- "id": "pattern-1780159888519-h-o34h",- "name": "docs: ruflo-workflows v0.4.0 native-Workflow orchestration + ADR-141 (taubench campaign)",+ "id": "pattern-1780159888518-b-vf8y",+ "name": "Merge feat/deterministic-tier1-codemods: deterministic Tier-1 codemods (ADR-143)","type": "history-commit",- "score": 0.1069707732645887+ "score": 0.5899985175159177,+ "cosineScore": 0.11334191563527016,+ "bm25Score": 0,+ "mmrScore": 0.29499925875795885},{- "id": "pattern-1780159888519-l-r8ns",- "name": "feat(statusline): configurable cost symbol and hide toggle (#2220)",+ "id": "pattern-1780161324689-22-p22q",+ "name": "Merge pull request #2101 from ruvnet/fix/2078-claudemd-coauthor-rule","type": "history-commit",- "score": 0.09827985114828856+ "score": 0.6,+ "cosineScore": 0.11741462257643585,+ "bm25Score": 0,+ "mmrScore": 0.27832122285750827},{- "id": "pattern-1780159888526-24-bn3i",- "name": "[Dream Cycle 2026-05-26] security: Indirect prompt injection critical gap vs OWASP ASI01 + intellige",- "type": "history-issue",- "score": 0.09399157789252553+ "id": "pattern-1780160010138-0-cefy",+ "name": "fix(intelligence): wire hooks_task-completed to recordTrajectory",+ "type": "history-commit",+ "score": 0.5450536587835099,+ "cosineScore": 0.09503990506628572,+ "bm25Score": 0,+ "mmrScore": 0.27101959503190853},{- "id": "pattern-1780159888525-1l-0t6s",- "name": "Add a `claude-code` provider β run `agent_execute`/`workflow_*`/`hive-mind` on a Claude Pro/Max subs",- "type": "history-issue",- "score": 0.0921915110194744+ "id": "pattern-1780161324698-3w-5c7h",+ "name": "feat(memory): #2061 ADR-125 Phase 5 β graceful retrieval degradation + FTS5 keyword fallback",+ "type": "history-commit",+ "score": 0.5750560127842083,+ "cosineScore": 0.1072571734146955,+ "bm25Score": 0,+ "mmrScore": 0.2557069865449775}]},{"query": "SQL injection migrate.ts table identifier","matched": true,+ "top1Relevant": false,+ "top3Relevant": false,+ "firstRelevantRank": -1,"topK": [{- "id": "pattern-1780159888524-18-ks7x",- "name": "feat(ci): #2132 β cross-platform audit for plugin hooks.json commands",+ "id": "pattern-1780161324703-51-v1b7",+ "name": "docs(browser): #2041 ADR-122 security audit + substrate benchmarks + announcement draft","type": "history-commit",- "score": 0.09711664650832315+ "score": 0.6510393622687756,+ "cosineScore": 0.019588773519894014,+ "bm25Score": 6.206553480055758,+ "mmrScore": 0.3255196811343878},{- "id": "pattern-1780159888526-1x-254e",- "name": "wizard: 'already initialized' detector false-positives on any `.claude/settings.json`",- "type": "history-issue",- "score": 0.09449409338072855+ "id": "pattern-1780161237211-19-7z58",+ "name": "feat(ci): #2132 β cross-platform audit for plugin hooks.json commands",+ "type": "history-commit",+ "score": 0.6121441415703033,+ "cosineScore": 0.09711664650832315,+ "bm25Score": 2.5401471715239787,+ "mmrScore": 0.3050938890510861},{- "id": "pattern-1780158992316-e-0arj",- "name": "SHABHNAM JAYAV 8Γ·58/π Rollback Incident: [Brief Description] not work",- "type": "history-issue",- "score": 0.09202215587572203+ "id": "pattern-1780161324691-2i-tm1b",+ "name": "Merge pull request #2090 from ruvnet/docs/adr-127-github-stack-modernization",+ "type": "history-commit",+ "score": 0.6,+ "cosineScore": 0.15794464752821977,+ "bm25Score": 0,+ "mmrScore": 0.3},{- "id": "pattern-1780159888525-1i-vt5g",- "name": "SHABHNAM JAYAV 8Γ·58/π Rollback Incident: [Brief Description] not work",- "type": "history-issue",- "score": 0.09202215587572203+ "id": "pattern-1780161237205-6-7kk3",+ "name": "chore(release): bump 3.10.12 β 3.10.13 + [email protected]",+ "type": "history-commit",+ "score": 0.5272599789668341,+ "cosineScore": -0.06928700588415701,+ "bm25Score": 7.325974016308216,+ "mmrScore": 0.2552980359566925},{- "id": "pattern-1780158992316-d-d89c",- "name": "[Dream Cycle 2026-05-30] performance: MV-HNSW 14Γ gap + LAMaS 38-46% latency + security,hive-mind sc",- "type": "history-issue",- "score": 0.08700450515219874+ "id": "pattern-1780161324689-21-o5x6",+ "name": "fix(smoke): #2042 β make providers smoke static-only (no build dependency)",+ "type": "history-commit",+ "score": 0.5486082159087727,+ "cosineScore": 0.13324219046664132,+ "bm25Score": 0,+ "mmrScore": 0.24417206666200053}]},{"query": "recall@k HNSW benchmark harness","matched": true,+ "top1Relevant": true,+ "top3Relevant": true,+ "firstRelevantRank": 1,"topK": [{- "id": "pattern-1780158992316-e-0arj",- "name": "SHABHNAM JAYAV 8Γ·58/π Rollback Incident: [Brief Description] not work",- "type": "history-issue",- "score": 0.16408597847506598+ "id": "pattern-1780161324693-2z-5x4a",+ "name": "feat(neural-trader): #48 β benchmark suite (signal, backtest, memory-recall)",+ "type": "history-commit",+ "score": 0.8018266090177799,+ "cosineScore": 0.08967419136034757,+ "bm25Score": 11.069668624696455,+ "mmrScore": 0.40091330450888996},{- "id": "pattern-1780159888525-1i-vt5g",- "name": "SHABHNAM JAYAV 8Γ·58/π Rollback Incident: [Brief Description] not work",- "type": "history-issue",- "score": 0.16408597847506598+ "id": "pattern-1780161237205-6-7kk3",+ "name": "chore(release): bump 3.10.12 β 3.10.13 + [email protected]",+ "type": "history-commit",+ "score": 0.6930655013125043,+ "cosineScore": 0.013638755437439873,+ "bm25Score": 12.520917564394331,+ "mmrScore": 0.34653275065625216},{- "id": "pattern-1780159888522-14-bf3x",- "name": "fix(security): #security-review v3.10 β bounds-check decodeEmbedding + env-var-precedence CI guard (",+ "id": "pattern-1780161324698-3y-v3g2",+ "name": "feat(memory): #2061 ADR-125 Phase 3 β persistent HNSW snapshot + restore","type": "history-commit",- "score": 0.12002098868824145+ "score": 0.5894330046400449,+ "cosineScore": 0.08630266804079345,+ "bm25Score": 4.636571427447146,+ "mmrScore": 0.2822710787648221},{- "id": "pattern-1780159888519-k-luhh",- "name": "fix: #2219 #2221 #2222 #2215 #2226 β memory/routing/statusline bug cluster (3.10.6) (#2227)",+ "id": "pattern-1780159888524-1d-x3m6",+ "name": "fix(bench): ADR-130 P6 β CI-friendly benchmark using single-session inserts","type": "history-commit",- "score": 0.10008728230638751+ "score": 0.5338765470062481,+ "cosineScore": 0.06474451013559841,+ "bm25Score": 4.274258657174938,+ "mmrScore": 0.26693827350312405},{- "id": "pattern-1780159888525-1s-i6i4",- "name": "Statusline shows `RuFlo V3.6` for global npm installs β statusline-generator version probe never che",+ "id": "pattern-1780158992316-e-0arj",+ "name": "SHABHNAM JAYAV 8Γ·58/π Rollback Incident: [Brief Description] not work","type": "history-issue",- "score": 0.08543416419702778+ "score": 0.6,+ "cosineScore": 0.16408597847506598,+ "bm25Score": 0,+ "mmrScore": 0.2563579067611487}]},{"query": "Q-learning encoder keyword block","matched": true,+ "top1Relevant": false,+ "top3Relevant": false,+ "firstRelevantRank": -1,"topK": [{- "id": "pattern-1780159888519-m-r8lt",- "name": "docs(adr): ADR-129 status β Accepted + Gap 2 implementation record (#2201)",+ "id": "pattern-1780161237205-9-48p8",+ "name": "chore(release): bump 3.10.10 β 3.10.11 (4-issue bug cluster)","type": "history-commit",- "score": 0.10618479339045693+ "score": 0.710559958201223,+ "cosineScore": 0.029870791404767158,+ "bm25Score": 13.600488646605163,+ "mmrScore": 0.3552799791006115},{- "id": "pattern-1780159888525-1s-i6i4",- "name": "Statusline shows `RuFlo V3.6` for global npm installs β statusline-generator version probe never che",- "type": "history-issue",- "score": 0.10389287629109815+ "id": "pattern-1780161324698-3u-c1g1",+ "name": "chore(release): publish 3.7.0-alpha.71 + @claude-flow/[email protected]",+ "type": "history-commit",+ "score": 0.6325338902596626,+ "cosineScore": 0.10356354291635028,+ "bm25Score": 3.099284643164071,+ "mmrScore": 0.30228090848432076},{- "id": "pattern-1780159888526-1w-jptg",- "name": "wizard: unconditional CLAUDE.md overwrite destroys project guidance",- "type": "history-issue",- "score": 0.09488348699864597+ "id": "pattern-1780161324688-1z-fu6u",+ "name": "chore(release): publish 3.7.0-alpha.78 β #2042 #2078 fixes",+ "type": "history-commit",+ "score": 0.6,+ "cosineScore": 0.12079199972660587,+ "bm25Score": 0,+ "mmrScore": 0.29654148767587263},{- "id": "pattern-1780158992306-1-jwma",- "name": "feat(intelligence): unified learning-stats aggregator (ADR-075, #2245 follow-up)",- "type": "history-commit",- "score": 0.09408856184596699+ "id": "pattern-1780161237213-1s-xof1",+ "name": "Statusline shows `RuFlo V3.6` for global npm installs β statusline-generator version probe never che",+ "type": "history-issue",+ "score": 0.61629844815063,+ "cosineScore": 0.10389287629109815,+ "bm25Score": 2.336254643968555,+ "mmrScore": 0.2783942537276099},{- "id": "pattern-1780159888516-1-gw0m",- "name": "feat(intelligence): unified learning-stats aggregator (ADR-075, #2245 follow-up)",+ "id": "pattern-1780159888519-m-r8lt",+ "name": "docs(adr): ADR-129 status β Accepted + Gap 2 implementation record (#2201)","type": "history-commit",- "score": 0.09408856184596699+ "score": 0.56928734499748,+ "cosineScore": 0.10618479339045693,+ "bm25Score": 0,+ "mmrScore": 0.27450268662114563}]},{"query": "security hardening crypto random IDs","matched": true,+ "top1Relevant": false,+ "top3Relevant": false,+ "firstRelevantRank": -1,"topK": [{- "id": "pattern-1780159888519-n-tlf5",- "name": "docs(gaia): submission reproducibility package (stable 34/53 config) (#2213)",+ "id": "pattern-1780161237205-6-7kk3",+ "name": "chore(release): bump 3.10.12 β 3.10.13 + [email protected]","type": "history-commit",- "score": 0.11382443239794945+ "score": 0.7292296778666876,+ "cosineScore": 0.03253431220092657,+ "bm25Score": 10.582768112698659,+ "mmrScore": 0.3646148389333438},{- "id": "pattern-1780159888525-1n-2zyq",- "name": "MCP servers orphan on every Claude Code restart β ~1 GB leaked, and a stale server silently serves t",- "type": "history-issue",- "score": 0.09565692842308834+ "id": "pattern-1780161324704-5d-ssyg",+ "name": "fix(hooks): #2017 β defensive pre-bash + new CI gate against silent-swallow shape",+ "type": "history-commit",+ "score": 0.6464283623857547,+ "cosineScore": 0.11760821356909369,+ "bm25Score": 2.8379359683329337,+ "mmrScore": 0.30529026970129103},{- "id": "pattern-1780159888522-13-1b3c",- "name": "feat: #2105 CLAUDE_FLOW_DB_PATH + #2058 team gateway checklist (#2146)",+ "id": "pattern-1780161324699-43-nlue",+ "name": "feat(memory): #2061 ADR-125 Phase 1 β MemoryService canonical entry point + deprecate UnifiedMemoryS","type": "history-commit",- "score": 0.09500525455993265+ "score": 0.6,+ "cosineScore": 0.14226244116789438,+ "bm25Score": 0,+ "mmrScore": 0.2970161868092974},{- "id": "pattern-1780159888519-l-r8ns",- "name": "feat(statusline): configurable cost symbol and hide toggle (#2220)",+ "id": "pattern-1780161324700-4b-ieca",+ "name": "Merge pull request #2050 from ruvnet/feat/ci-supply-chain-hardening","type": "history-commit",- "score": 0.09439222856751739+ "score": 0.592484117643453,+ "cosineScore": 0.052350663373743465,+ "bm25Score": 5.6711645256634045,+ "mmrScore": 0.28617992493599675},{- "id": "pattern-1780159888520-q-3lmr",- "name": "fix(init): #2206 #2207 #2208 β wizard server key, init detector, CLAUDE.md backup (#2214)",+ "id": "pattern-1780161324693-33-7swi",+ "name": "fix(init): #2078 β use no-reply bot email for opt-in Co-Authored-By","type": "history-commit",- "score": 0.08303717864476265+ "score": 0.5500381877424747,+ "cosineScore": 0.12201570165957405,+ "bm25Score": 0,+ "mmrScore": 0.2532795338785324}]}diff --git a/package-lock.json b/package-lock.jsonindex fd11924a3c..c40c6e6d53 100644--- a/package-lock.json+++ b/package-lock.json@@ -1,12 +1,12 @@{"name": "claude-flow",- "version": "3.10.17",+ "version": "3.10.18","lockfileVersion": 3,"requires": true,"packages": {"": {"name": "claude-flow",- "version": "3.10.17",+ "version": "3.10.18","license": "MIT","dependencies": {"@claude-flow/cli-core": "^3.7.0-alpha.5",diff --git a/package.json b/package.jsonindex 48fbe611be..81b97f70d7 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "claude-flow",- "version": "3.10.17",+ "version": "3.10.18","description": "Ruflo - Enterprise AI agent orchestration for Claude Code. Deploy 60+ specialized agents in coordinated swarms with self-learning, fault-tolerant consensus, vector memory, and MCP integration","main": "dist/index.js","type": "module",diff --git a/ruflo/package.json b/ruflo/package.jsonindex 545e5ff1b1..cc22d3f50d 100644--- a/ruflo/package.json+++ b/ruflo/package.json@@ -1,6 +1,6 @@{"name": "ruflo",- "version": "3.10.17",+ "version": "3.10.18","description": "Ruflo - Enterprise AI agent orchestration platform. Deploy 60+ specialized agents in coordinated swarms with self-learning, fault-tolerant consensus, vector memory, and MCP integration","main": "bin/ruflo.js","type": "module",diff --git a/v3/@claude-flow/cli/__tests__/hybrid-retrieval.test.ts b/v3/@claude-flow/cli/__tests__/hybrid-retrieval.test.tsnew file mode 100644index 0000000000..838d764fdf--- /dev/null+++ b/v3/@claude-flow/cli/__tests__/hybrid-retrieval.test.ts@@ -0,0 +1,177 @@+// Hybrid retrieval β pure-function tests (ADR-078).+//+// These tests do not touch the neural store, embeddings, or any I/O. They+// pin the behaviour of the building blocks so the wiring in neural-tools.ts+// can rely on them.++import { describe, it, expect } from 'vitest';+import {+ tokenize,+ buildCorpusStats,+ bm25Score,+ normalise,+ hybridScores,+ cosineSim,+ mmrRerank,+} from '../src/memory/hybrid-retrieval.js';++describe('tokenize', () => {+ it('lowercases + splits on non-alphanumeric', () => {+ const t = tokenize('Refactor src/Auth/middleware.ts to use jwt-verify!');+ // Keeps '/' '-' '.' for paths/identifiers; drops punctuation.+ expect(t).toContain('refactor');+ expect(t).toContain('src/auth/middleware.ts');+ expect(t).toContain('jwt-verify');+ });++ it('drops stopwords and tokens shorter than 3 chars', () => {+ const t = tokenize('the cat is on a mat');+ // 'the','is','on','a' are stopwords; 'cat'/'mat' length 3 stay.+ expect(t).not.toContain('the');+ expect(t).not.toContain('is');+ expect(t).toContain('cat');+ expect(t).toContain('mat');+ });++ it('handles empty / null-ish input', () => {+ expect(tokenize('')).toEqual([]);+ });+});++describe('buildCorpusStats + bm25Score', () => {+ const docs = [+ tokenize('refactor authentication middleware src/auth.ts'),+ tokenize('add unit tests for src/auth.ts middleware'),+ tokenize('bump release version chore'),+ tokenize('fix sql injection in src/db/migrate.ts'),+ ];+ const stats = buildCorpusStats(docs);++ it('counts documents and averages doc length', () => {+ expect(stats.N).toBe(4);+ expect(stats.avgDocLen).toBeGreaterThan(0);+ });++ it('builds IDF β rare tokens score higher than common ones', () => {+ // 'src/auth.ts' appears in 2/4 docs; 'sql' appears in 1/4 β 'sql' is rarer.+ const idfSql = stats.idf.get('sql') ?? 0;+ const idfAuth = stats.idf.get('src/auth.ts') ?? 0;+ expect(idfSql).toBeGreaterThan(idfAuth);+ });++ it('ranks the relevant doc highest for an exact-token query', () => {+ const q = tokenize('sql injection migrate');+ const scores = docs.map((d) => bm25Score(q, d, stats));+ const top = scores.indexOf(Math.max(...scores));+ expect(top).toBe(3); // The "fix sql injection" doc+ });++ it('returns 0 when there is no token overlap', () => {+ const q = tokenize('quantum computing breakthrough');+ const scores = docs.map((d) => bm25Score(q, d, stats));+ expect(Math.max(...scores)).toBe(0);+ });+});++describe('normalise', () => {+ it('maps to [0,1] preserving relative order', () => {+ const out = normalise([1, 3, 5, 7]);+ expect(out[0]).toBe(0);+ expect(out[3]).toBe(1);+ expect(out[1]).toBeLessThan(out[2]);+ });++ it('handles constant vectors without divide-by-zero', () => {+ const out = normalise([0.5, 0.5, 0.5]);+ expect(out).toEqual([0.5, 0.5, 0.5]);+ });++ it('handles empty input', () => {+ expect(normalise([])).toEqual([]);+ });+});++describe('hybridScores', () => {+ it('linearly combines cosine and BM25 on normalised scales', () => {+ const cosine = [0.1, 0.5, 0.9];+ const bm25 = [3.0, 1.0, 0.0];+ const hybrid = hybridScores(cosine, bm25, 0.5);+ // index 0 has low cosine but high BM25 β should beat index 1's middling both.+ expect(hybrid[0]).toBeGreaterThan(hybrid[1]);+ // index 2 has high cosine but 0 BM25 β roughly comparable to index 0.+ expect(hybrid[2]).toBeGreaterThan(0);+ });++ it('alpha=1 collapses to cosine ordering', () => {+ const cosine = [0.1, 0.9, 0.5];+ const bm25 = [10, 0, 5];+ const hybrid = hybridScores(cosine, bm25, 1.0);+ const order = hybrid.map((s, i) => [i, s]).sort((a, b) => (b[1] as number) - (a[1] as number)).map((x) => x[0]);+ expect(order[0]).toBe(1); // highest cosine wins+ });++ it('alpha=0 collapses to BM25 ordering', () => {+ const cosine = [0.9, 0.1, 0.5];+ const bm25 = [0, 10, 5];+ const hybrid = hybridScores(cosine, bm25, 0.0);+ const order = hybrid.map((s, i) => [i, s]).sort((a, b) => (b[1] as number) - (a[1] as number)).map((x) => x[0]);+ expect(order[0]).toBe(1); // highest BM25 wins+ });++ it('throws on mismatched lengths', () => {+ expect(() => hybridScores([0.1], [0.2, 0.3], 0.5)).toThrow();+ });+});++describe('cosineSim', () => {+ it('returns 1 for identical vectors', () => {+ expect(cosineSim([1, 0, 0], [1, 0, 0])).toBeCloseTo(1, 6);+ });+ it('returns 0 for orthogonal vectors', () => {+ expect(cosineSim([1, 0], [0, 1])).toBe(0);+ });+ it('returns 0 on zero-norm input', () => {+ expect(cosineSim([0, 0, 0], [1, 1, 1])).toBe(0);+ });+});++describe('mmrRerank', () => {+ it('picks the top-relevance candidate first', () => {+ const cands = [+ { id: 'a', embedding: [1, 0, 0], relevance: 0.5 },+ { id: 'b', embedding: [0, 1, 0], relevance: 0.9 },+ { id: 'c', embedding: [0, 0, 1], relevance: 0.3 },+ ];+ const out = mmrRerank(cands, 3, 0.5);+ expect(out[0].id).toBe('b');+ });++ it('suppresses near-duplicates when lambda<1', () => {+ // Two near-identical embeddings with high relevance, one distinct lower one.+ const cands = [+ { id: 'dup1', embedding: [1.0, 0.0, 0.0], relevance: 0.95 },+ { id: 'dup2', embedding: [0.99, 0.01, 0.0], relevance: 0.93 },+ { id: 'diverse', embedding: [0.0, 0.0, 1.0], relevance: 0.6 },+ ];+ const out = mmrRerank(cands, 2, 0.3); // bias hard toward diversity+ expect(out[0].id).toBe('dup1');+ // With strong diversity bias the second pick should be the diverse one,+ // not the near-duplicate.+ expect(out[1].id).toBe('diverse');+ });++ it('lambda=1.0 ignores diversity (pure relevance ordering)', () => {+ const cands = [+ { id: 'dup1', embedding: [1.0, 0.0], relevance: 0.95 },+ { id: 'dup2', embedding: [0.99, 0.01], relevance: 0.93 },+ { id: 'diverse', embedding: [0.0, 1.0], relevance: 0.6 },+ ];+ const out = mmrRerank(cands, 2, 1.0);+ expect(out.map((o) => o.id)).toEqual(['dup1', 'dup2']);+ });++ it('handles empty and k=0', () => {+ expect(mmrRerank([], 3, 0.5)).toEqual([]);+ expect(mmrRerank([{ id: 'x', embedding: [1], relevance: 1 }], 0, 0.5)).toEqual([]);+ });+});diff --git a/v3/@claude-flow/cli/__tests__/pretrain-from-github.test.ts b/v3/@claude-flow/cli/__tests__/pretrain-from-github.test.tsindex ab6a22e199..2a7681d99d 100644--- a/v3/@claude-flow/cli/__tests__/pretrain-from-github.test.ts+++ b/v3/@claude-flow/cli/__tests__/pretrain-from-github.test.ts@@ -21,7 +21,11 @@ import {getUnifiedLearningStats,flushIntelligenceStats,} from '../src/memory/intelligence.js';-import { storeNeuralPatterns, getNeuralStoreStats } from '../src/mcp-tools/neural-tools.js';+import {+ storeNeuralPatterns,+ getNeuralStoreStats,+ neuralTools,+} from '../src/mcp-tools/neural-tools.js';// Fixture β a tiny mock "history" shaped like real ruflo commits + issues.// Intentionally small so this stays fast. The shape (subject + body + verdict)@@ -131,4 +135,41 @@ describe('pretrain-from-github wiring guard (#2245 regression)', () => {const distilled = distillAndSerialise(`${item.subject}\n\n${item.body}`);expect(distilled).toMatch(/2245|self-learning|persists/i);});++ // ADR-078 β hybrid retrieval should be at least as discriminative as cosine+ // alone on an exact-keyword query. We don't require a strict inequality+ // because on a 5-item fixture both modes can return the same top-1.+ it('hybrid search returns a non-empty result and respects mode parameter', async () => {+ const tool = neuralTools.find((t) => t.name === 'neural_patterns');+ expect(tool).toBeDefined();++ const hybrid = await tool!.handler({+ action: 'search',+ query: 'structured distillation 4-field schema',+ mode: 'hybrid',+ limit: 3,+ });+ expect(hybrid.mode).toBe('hybrid');+ expect(Array.isArray(hybrid.results)).toBe(true);+ expect(hybrid.results.length).toBeGreaterThan(0);+ // Top-1 of a query targeting commit-aaa3 (Structured Distillation, ADR-076)+ // should mention either 'structured', 'distillation', or 'ADR-076'.+ const top1Name = String(hybrid.results[0].name).toLowerCase();+ expect(top1Name).toMatch(/structured|distillation|adr-076|076/);+ // Hybrid path returns the extra ADR-078 fields.+ expect(hybrid.results[0]).toHaveProperty('hybridScore');+ expect(hybrid.results[0]).toHaveProperty('cosineScore');+ expect(hybrid.results[0]).toHaveProperty('bm25Score');++ const cosine = await tool!.handler({+ action: 'search',+ query: 'structured distillation 4-field schema',+ mode: 'cosine',+ limit: 3,+ });+ expect(cosine.mode).toBe('cosine');+ expect(Array.isArray(cosine.results)).toBe(true);+ // Cosine path does NOT include the hybrid breakdown.+ expect(cosine.results[0]).not.toHaveProperty('hybridScore');+ });});diff --git a/v3/@claude-flow/cli/package.json b/v3/@claude-flow/cli/package.jsonindex a18950baa8..01e8306b45 100644--- a/v3/@claude-flow/cli/package.json+++ b/v3/@claude-flow/cli/package.json@@ -1,6 +1,6 @@{"name": "@claude-flow/cli",- "version": "3.10.17",+ "version": "3.10.18","type": "module","description": "Ruflo CLI - Enterprise AI agent orchestration with 60+ specialized agents, swarm coordination, MCP server, self-learning hooks, and vector memory for Claude Code","main": "dist/src/index.js",diff --git a/v3/@claude-flow/cli/scripts/benchmark-pretrained-retrieval.mjs b/v3/@claude-flow/cli/scripts/benchmark-pretrained-retrieval.mjsindex 7068425ff0..f700cef5f3 100644--- a/v3/@claude-flow/cli/scripts/benchmark-pretrained-retrieval.mjs+++ b/v3/@claude-flow/cli/scripts/benchmark-pretrained-retrieval.mjs@@ -21,19 +21,20 @@ const REPO_ROOT = resolve(SCRIPT_DIR, '../../../..');const RUNS_DIR = join(REPO_ROOT, 'docs', 'benchmarks', 'runs');// Real ruflo-history-shaped queries. Each one targets a concept that should-// have been seen during pretrain (Opus 4.8 fix, self-learning wiring,-// codemod engine, etc.).+// have been seen during pretrain. `expect` is a regex matched against the+// result's `name` field (commit subject or issue title) to score relevance.+// A result is "relevant" if its name matches the expect regex (case-insensitive).const QUERIES = [- 'how was the Opus model alias fixed',- 'self-learning wiring task-completed pretrain',- 'deterministic codemod engine var-to-const',- 'MCP server orphan leak parent-death',- 'unified learning stats aggregator',- 'structured distillation 4-field schema',- 'SQL injection migrate.ts table identifier',- 'recall@k HNSW benchmark harness',- 'Q-learning encoder keyword block',- 'security hardening crypto random IDs',+ { q: 'how was the Opus model alias fixed', expect: /opus|2232|model.*alias|4\.8/i },+ { q: 'self-learning wiring task-completed pretrain', expect: /self.?learning|task.?completed|pretrain|2245|074/i },+ { q: 'deterministic codemod engine var-to-const', expect: /codemod|var.?to.?const|143|deterministic/i },+ { q: 'MCP server orphan leak parent-death', expect: /mcp.*orphan|orphan.*mcp|parent.?death|leak/i },+ { q: 'unified learning stats aggregator', expect: /unified|stats|aggregator|075/i },+ { q: 'structured distillation 4-field schema', expect: /distillation|structured|076|4.?field/i },+ { q: 'SQL injection migrate.ts table identifier', expect: /sql.?injection|migrate|table|identifier/i },+ { q: 'recall@k HNSW benchmark harness', expect: /recall|hnsw|benchmark|harness/i },+ { q: 'Q-learning encoder keyword block', expect: /q.?learning|encoder|keyword|2239/i },+ { q: 'security hardening crypto random IDs', expect: /security|hardening|crypto|random/i },];const TOP_K = 5;@@ -51,35 +52,97 @@ async function main() {process.exit(2);}- // Β§2 β run each query through neural_patterns search. The tool's search- // action runs cosine similarity against the stored embeddings.+ // Β§2 β A/B hybrid vs cosine-only (HYBRID=0 forces pre-3.10.18 behaviour).+ // Default runs hybrid (cosine + BM25 + MMR per ADR-078).const listTool = neural.neuralTools.find((t) => t.name === 'neural_patterns');+ const mode = process.env.HYBRID === '0' ? 'cosine' : 'hybrid';++ // top-1-uniqueness β fraction of queries whose top-1 result is NOT the+ // same pattern ID as another query's top-1. Catches the "everyone gets+ // the same generic top-1" failure mode.+ const top1Ids = new Map();+const tQuery0 = performance.now();const results = [];- for (const q of QUERIES) {- const r = await listTool.handler({ action: 'search', query: q });+ for (const { q, expect } of QUERIES) {+ const r = await listTool.handler({ action: 'search', query: q, mode, limit: TOP_K });const matches = (r.patterns || r.results || r.matches || []).slice(0, TOP_K);+ if (matches.length > 0) {+ const top1 = matches[0].id;+ top1Ids.set(top1, (top1Ids.get(top1) ?? 0) + 1);+ }+ // ADR-078 relevance: does the top-1/top-3 name match the expect regex?+ const top1Name = matches[0]?.name ?? '';+ const top1Relevant = expect.test(top1Name);+ let top3Relevant = false;+ let firstRelevantRank = -1;+ for (let i = 0; i < Math.min(matches.length, 3); i++) {+ if (expect.test(matches[i]?.name ?? '')) {+ top3Relevant = true;+ if (firstRelevantRank === -1) firstRelevantRank = i + 1;+ break;+ }+ }results.push({query: q,matched: matches.length > 0,+ top1Relevant,+ top3Relevant,+ firstRelevantRank,topK: matches.map((m) => ({id: m.id,name: m.name?.slice(0, 100),type: m.type,score: m.score ?? m.similarity,+ cosineScore: m.cosineScore,+ bm25Score: m.bm25Score,+ mmrScore: m.mmrScore,})),});}const queryMs = performance.now() - tQuery0;const matchedQueries = results.filter((r) => r.matched).length;+ const top1Hits = results.filter((r) => r.top1Relevant).length;+ const top3Hits = results.filter((r) => r.top3Relevant).length;+ const ranks = results.filter((r) => r.firstRelevantRank > 0).map((r) => r.firstRelevantRank);+ // MRR over the top-3 window (rank-of-first-relevant). Queries with no+ // relevant result in top-3 contribute 0 to the mean.+ const mrr3 = QUERIES.length > 0+ ? Number((ranks.reduce((s, r) => s + 1 / r, 0) / QUERIES.length).toFixed(4))+ : 0;+ // top-1 collision: number of distinct top-1 IDs over the query count.+ const uniqueTop1 = top1Ids.size;+ const top1Diversity = Number((uniqueTop1 / QUERIES.length).toFixed(4));++ // top-3 redundancy: average fraction of top-3 results that are duplicates+ // of the same pattern within a single query (should be 0 β but if the same+ // ID appears twice in top-3 we surface it).+ let dupCount = 0, top3Slots = 0;+ for (const r of results) {+ const ids = r.topK.slice(0, 3).map((m) => m.id);+ top3Slots += ids.length;+ const seen = new Set();+ for (const id of ids) {+ if (seen.has(id)) dupCount++;+ seen.add(id);+ }+ }+ const top3DupRate = top3Slots > 0 ? Number((dupCount / top3Slots).toFixed(4)) : 0;+const summary = {runAt: new Date().toISOString(),benchmark: 'pretrained-retrieval',+ mode, // ADR-078: which retrieval path was usedstoreSize: total,queries: QUERIES.length,matchedQueries,matchRate: Number((matchedQueries / QUERIES.length).toFixed(4)),+ top1HitRate: Number((top1Hits / QUERIES.length).toFixed(4)), // ADR-078: relevance, not just any-match+ top3HitRate: Number((top3Hits / QUERIES.length).toFixed(4)),+ mrr3, // mean reciprocal rank over top-3+ top1Diversity, // 1.0 = every query gets a distinct top-1+ top3DupRate, // 0.0 = no duplicate IDs inside any top-3avgQueryLatencyMs: Number((queryMs / QUERIES.length).toFixed(2)),totalQueryMs: Number(queryMs.toFixed(2)),results,@@ -90,9 +153,15 @@ async function main() {console.log(JSON.stringify(summary, null, 2));} else {console.log(`# Pretrained-retrieval benchmark β proof of learning`);+ console.log(`Mode: ${mode}${mode === 'hybrid' ? ' (cosine + BM25 + MMR, ADR-078)' : ' (cosine-only, pre-3.10.18)'}`);console.log(`Store size: ${total} patterns`);console.log(`Queries: ${QUERIES.length}`);console.log(`Match rate: ${(summary.matchRate * 100).toFixed(0)}% (${matchedQueries}/${QUERIES.length})`);+ console.log(`Top-1 hit rate (RELEVANCE): ${(summary.top1HitRate * 100).toFixed(0)}% (${top1Hits}/${QUERIES.length})`);+ console.log(`Top-3 hit rate (RELEVANCE): ${(summary.top3HitRate * 100).toFixed(0)}% (${top3Hits}/${QUERIES.length})`);+ console.log(`MRR@3: ${summary.mrr3}`);+ console.log(`Top-1 diversity: ${(summary.top1Diversity * 100).toFixed(0)}% (${uniqueTop1} distinct top-1 IDs across ${QUERIES.length} queries)`);+ console.log(`Top-3 dup rate: ${(summary.top3DupRate * 100).toFixed(0)}%`);console.log(`Avg query latency: ${summary.avgQueryLatencyMs} ms`);console.log('');for (const r of results) {diff --git a/v3/@claude-flow/cli/scripts/pretrain-from-github.mjs b/v3/@claude-flow/cli/scripts/pretrain-from-github.mjsindex 7699377867..9e6ccefb67 100644--- a/v3/@claude-flow/cli/scripts/pretrain-from-github.mjs+++ b/v3/@claude-flow/cli/scripts/pretrain-from-github.mjs@@ -43,29 +43,110 @@ const SOURCE = process.env.SOURCE || 'all'; // 'all' | 'git' | 'issues'// Harvesters// ---------------------------------------------------------------------------+// ADR-078 outcome signal β classify each commit by whether it was reverted,+// hotfix-followed, or stuck. Operates on the harvested-commits window plus a+// wider lookahead window for revert/hotfix detection.+//+// Verdicts emitted:+// success β landed cleanly, no later commit reverted or fixed it+// reverted β a later commit subject starts with `Revert "<this subject>"`+// hotfixed β a later commit (within HOTFIX_WINDOW_COMMITS) shares >=50%+// of the same touched files AND subject contains fix|hotfix|patch+//+// The "later" direction is git-log order (newest first β we look at older+// indices in the lookahead, which are NEWER commits chronologically).+const HOTFIX_WINDOW_COMMITS = Number(process.env.HOTFIX_WINDOW_COMMITS) || 20;+const HOTFIX_FILE_OVERLAP = Number(process.env.HOTFIX_FILE_OVERLAP) || 0.5;+const HOTFIX_KEYWORDS = /\b(fix|hotfix|patch|revert|bugfix|fixup)\b/i;+function harvestCommits(n) {if (SOURCE === 'issues') return [];const fmt = '%H%x00%s%x00%b%x01';+ // Pull a wider window so we have lookahead for revert/hotfix detection+ // without changing the trained set size n.+ const lookahead = HOTFIX_WINDOW_COMMITS;const raw = execSync(- `git log --pretty=format:'${fmt}' -n ${n} 2>/dev/null`,+ `git log --pretty=format:'${fmt}' -n ${n + lookahead} 2>/dev/null`,{ cwd: REPO_ROOT, encoding: 'utf-8', maxBuffer: 64 * 1024 * 1024 },);- const entries = [];++ // Parse all blocks (window + lookahead).+ const all = [];for (const block of raw.split('\x01')) {if (!block.trim()) continue;const [sha, subject, body] = block.split('\x00');if (!sha || !subject) continue;+ all.push({ sha, subject: subject.trim(), body: (body || '').trim() });+ }++ // For each commit in the trained slice, get its touched files.+ const filesCache = new Map();+ const filesOf = (sha) => {+ if (filesCache.has(sha)) return filesCache.get(sha);+ try {+ const out = execSync(+ `git show --pretty=format: --name-only ${sha} 2>/dev/null`,+ { cwd: REPO_ROOT, encoding: 'utf-8', maxBuffer: 8 * 1024 * 1024 },+ ).trim().split('\n').filter(Boolean);+ filesCache.set(sha, new Set(out));+ return filesCache.get(sha);+ } catch {+ filesCache.set(sha, new Set());+ return filesCache.get(sha);+ }+ };++ // Build a revert-target map: any commit whose subject starts with+ // `Revert "<X>"` flags X as reverted.+ const revertedSubjects = new Set();+ for (const c of all) {+ const m = c.subject.match(/^Revert\s+"(.+?)"/);+ if (m) revertedSubjects.add(m[1].trim());+ }++ const trained = all.slice(0, n);+ const entries = [];+ for (let i = 0; i < trained.length; i++) {+ const c = trained[i];+ const myFiles = filesOf(c.sha);++ let verdict = 'success';+ let outcomeNote = null;++ // Reverted? Check if any later (lower-index = newer) commit reverted us.+ if (revertedSubjects.has(c.subject)) {+ verdict = 'reverted';+ outcomeNote = 'subject reverted by a later commit';+ } else {+ // Hotfixed? A later commit within HOTFIX_WINDOW_COMMITS shares files+ // AND has fix/hotfix in its subject.+ const start = Math.max(0, i - HOTFIX_WINDOW_COMMITS);+ for (let j = start; j < i; j++) {+ const later = trained[j];+ if (!HOTFIX_KEYWORDS.test(later.subject)) continue;+ const laterFiles = filesOf(later.sha);+ if (laterFiles.size === 0 || myFiles.size === 0) continue;+ let overlap = 0;+ for (const f of laterFiles) if (myFiles.has(f)) overlap++;+ // Use min() so a small targeted fix on a big change still triggers+ // (semantic: "β₯half of the smaller commit's files overlap").+ const overlapFrac = overlap / Math.min(laterFiles.size, myFiles.size);+ if (overlapFrac >= HOTFIX_FILE_OVERLAP) {+ verdict = 'hotfixed';+ outcomeNote = `${(overlapFrac * 100).toFixed(0)}% file overlap with later fix ${later.sha.slice(0, 8)}`;+ break;+ }+ }+ }+entries.push({source: 'commit',- id: `commit-${sha.slice(0, 12)}`,- subject: subject.trim(),- body: (body || '').trim(),- // The "verdict" of a commit isn't directly known. Treat all commits as- // success β we're learning from intent, not outcome. Failures are- // captured separately via the issue stream (closed issues are- // success; open issues are partial).- verdict: 'success',- content: `${subject.trim()}\n\n${(body || '').trim()}`.slice(0, 8192),+ id: `commit-${c.sha.slice(0, 12)}`,+ subject: c.subject,+ body: c.body,+ verdict,+ outcomeNote,+ content: `${c.subject}\n\n${c.body}`.slice(0, 8192),});}return entries;@@ -126,7 +207,19 @@ async function main() {console.log(`Harvested: ${commits.length} commits + ${issues.length} issues = ${items.length} trajectories (${harvestMs.toFixed(0)} ms)`);}- // Β§3 β feed each item through the trajectory pipeline.+ // Β§3 β feed each item through the trajectory pipeline. The harvester+ // emits one of: success | partial | reverted | hotfixed. We map this to+ // the trajectory pipeline's binary verdict {success, partial} since the+ // pipeline doesn't accept arbitrary strings β but we preserve the original+ // outcome in metadata + the summary's verdictMix so the signal isn't lost.+ const verdictToPipeline = (v) => {+ if (v === 'success') return 'success';+ if (v === 'partial') return 'partial';+ if (v === 'reverted') return 'partial'; // a revert is the strongest "this was wrong" signal we have+ if (v === 'hotfixed') return 'partial'; // a same-files fix-followup is "needs adjustment"+ return 'success';+ };+ const verdictMix = { success: 0, partial: 0, reverted: 0, hotfixed: 0 };const tFeed0 = performance.now();let trained = 0;let failed = 0;@@ -134,6 +227,7 @@ async function main() {for (const item of items) {try {const distilled = distillAndSerialise(item.content);+ verdictMix[item.verdict] = (verdictMix[item.verdict] ?? 0) + 1;await intel.recordTrajectory([{type: 'result',@@ -142,10 +236,12 @@ async function main() {source: item.source,id: item.id,subject: item.subject.slice(0, 200),+ outcomeVerdict: item.verdict,+ outcomeNote: item.outcomeNote ?? null,},timestamp: Date.now(),}],- item.verdict,+ verdictToPipeline(item.verdict),);trained++;} catch (err) {@@ -201,6 +297,7 @@ async function main() {avgLatencyMs: items.length > 0 ? Number((feedMs / items.length).toFixed(2)) : 0,totalMs: Number(feedMs.toFixed(2)),sampleFailures: failures,+ verdictMix, // ADR-078 outcome signal β counts per harvested verdict},seedNeuralStore: {stored: seedResult.stored,@@ -231,6 +328,7 @@ async function main() {console.log(`Failed: ${failed}`);console.log(`Avg latency per trajectory: ${summary.feed.avgLatencyMs} ms`);console.log(`Neural store seeded: ${seedResult.stored}/${seedResult.total}`);+ console.log(`Verdict mix: success=${verdictMix.success} partial=${verdictMix.partial} reverted=${verdictMix.reverted} hotfixed=${verdictMix.hotfixed}`);console.log(`Overall: ${summary.passed ? 'β PASSED' : 'β οΈ partial'}`);if (unified1.consistency.notes.length > 0) {console.log(`\nConsistency notes:`);diff --git a/v3/@claude-flow/cli/src/mcp-tools/neural-tools.ts b/v3/@claude-flow/cli/src/mcp-tools/neural-tools.tsindex a057dc4b58..402e994844 100644--- a/v3/@claude-flow/cli/src/mcp-tools/neural-tools.ts+++ b/v3/@claude-flow/cli/src/mcp-tools/neural-tools.ts@@ -97,6 +97,10 @@ interface Pattern {name: string;type: string;embedding: number[];+ /** Source text the embedding was built from. Cap 4096 chars. Used for+ * BM25 in hybrid retrieval (ADR-078). Optional for backwards compat β+ * pre-3.10.18 patterns fall back to `name` for BM25 tokenisation. */+ content?: string;metadata: Record<string, unknown>;createdAt: string;usageCount: number;@@ -187,13 +191,15 @@ export async function storeNeuralPatterns(items: Array<{let stored = 0;for (const item of items) {if (!item.name || !item.type) continue;- const embedding = await generateEmbedding(item.content ?? item.name);+ const sourceText = item.content ?? item.name;+ const embedding = await generateEmbedding(sourceText);const id = `pattern-${Date.now()}-${stored.toString(36)}-${Math.random().toString(36).slice(2, 6)}`;store.patterns[id] = {id,name: String(item.name).slice(0, 200),type: String(item.type).slice(0, 64),embedding,+ content: typeof sourceText === 'string' ? sourceText.slice(0, 4096) : undefined,metadata: item.metadata ?? {},createdAt: new Date().toISOString(),usageCount: 0,@@ -484,7 +490,12 @@ export const neuralTools: MCPTool[] = [patternId: { type: 'string', description: 'Pattern ID' },name: { type: 'string', description: 'Pattern name' },type: { type: 'string', description: 'Pattern type' },+ content: { type: 'string', description: 'Pattern source text (used for BM25 in hybrid search; falls back to name)' },query: { type: 'string', description: 'Search query' },+ limit: { type: 'number', description: 'Top-K results to return (default 10, max 100)' },+ mode: { type: 'string', enum: ['hybrid', 'cosine'], description: 'Search mode β hybrid (cosine+BM25+MMR, default) or cosine (pre-3.10.18 behaviour, for A/B)' },+ alpha: { type: 'number', description: 'Hybrid: cosine weight in [0,1]; (1-Ξ±) is BM25 weight (default 0.6)' },+ mmrLambda: { type: 'number', description: 'Hybrid: MMR balance β 1.0 = pure relevance, 0.0 = pure diversity (default 0.5)' },data: { type: 'object', description: 'Pattern data' },},},@@ -525,15 +536,17 @@ export const neuralTools: MCPTool[] = [if (action === 'store') {const patternId = `pattern-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`;const patternName = (input.name as string) || 'Unnamed pattern';+ const patternContent = (input.content as string) ?? patternName;- // Generate embedding from pattern name/content- const embedding = await generateEmbedding(patternName, 384);+ // Generate embedding from pattern content (falls back to name).+ const embedding = await generateEmbedding(patternContent, 384);const pattern: Pattern = {id: patternId,name: patternName,type: (input.type as string) || 'general',embedding,+ content: typeof patternContent === 'string' ? patternContent.slice(0, 4096) : undefined,metadata: (input.data as Record<string, unknown>) || {},createdAt: new Date().toISOString(),usageCount: 0,@@ -557,32 +570,83 @@ export const neuralTools: MCPTool[] = [if (action === 'search') {const query = input.query as string;+ const k = Math.min(Math.max(Number(input.limit ?? input.topK ?? 10), 1), 100);+ // ADR-078 hybrid retrieval controls. Cosine-only mode preserves the+ // pre-3.10.18 behaviour for A/B tests via {mode:'cosine'}; default is+ // hybrid (cosine + BM25 + MMR).+ const mode = String(input.mode ?? 'hybrid');+ const alpha = Number(input.alpha ?? 0.6); // cosine weight; (1-Ξ±) is BM25+ const mmrLambda = Number(input.mmrLambda ?? 0.5); // 1 = pure relevance, 0 = pure diversity++ const { tokenize, buildCorpusStats, bm25Score, hybridScores, mmrRerank } =+ await import('../memory/hybrid-retrieval.js');- // Generate query embedding for real similarity searchconst queryEmbedding = await generateEmbedding(query, 384);+ const patterns = Object.values(store.patterns);++ // Compute cosine for every pattern (this is what the old path did).+ const cosineArr = patterns.map((p) => cosineSimilarity(queryEmbedding, p.embedding));++ if (mode === 'cosine') {+ const ranked = patterns+ .map((p, i) => ({ ...p, similarity: cosineArr[i] }))+ .sort((a, b) => b.similarity - a.similarity)+ .slice(0, k);+ return {+ _realSimilarity: true,+ _realEmbedding: !!realEmbeddings,+ _embeddingSource: embeddingServiceName,+ embeddingProvider: embeddingServiceName,+ mode: 'cosine',+ query,+ results: ranked.map((r) => ({ id: r.id, name: r.name, type: r.type, similarity: r.similarity })),+ total: ranked.length,+ };+ }- // Calculate REAL cosine similarity against stored patterns- const results = Object.values(store.patterns)- .map(p => ({- ...p,- similarity: cosineSimilarity(queryEmbedding, p.embedding),- }))- .sort((a, b) => b.similarity - a.similarity)- .slice(0, 10);+ // Hybrid path β BM25 over the pattern's content (falls back to name+ // for pre-3.10.18 patterns missing the field).+ const docs = patterns.map((p) => tokenize(p.content ?? p.name));+ const stats = buildCorpusStats(docs);+ const queryTokens = tokenize(query);+ const bm25Arr = docs.map((d) => bm25Score(queryTokens, d, stats));+ const hybridArr = hybridScores(cosineArr, bm25Arr, alpha);++ // MMR diversity over top-K*3 hybrid candidates β top-K final.+ const prelim = patterns+ .map((p, i) => ({ p, hybrid: hybridArr[i], cosine: cosineArr[i], bm25: bm25Arr[i] }))+ .sort((a, b) => b.hybrid - a.hybrid)+ .slice(0, Math.min(k * 3, patterns.length));++ const mmrCandidates = prelim.map(({ p, hybrid, cosine, bm25 }) => ({+ id: p.id, name: p.name, type: p.type,+ embedding: p.embedding,+ relevance: hybrid,+ _cosine: cosine,+ _bm25: bm25,+ }));+ const picked = mmrRerank(mmrCandidates, k, mmrLambda);return {_realSimilarity: true,_realEmbedding: !!realEmbeddings,_embeddingSource: embeddingServiceName,embeddingProvider: embeddingServiceName,+ mode: 'hybrid',+ alpha,+ mmrLambda,query,- results: results.map(r => ({+ results: picked.map((r) => ({id: r.id,name: r.name,type: r.type,- similarity: r.similarity,+ similarity: r.relevance, // exposed as `similarity` for back-compat+ hybridScore: r.relevance,+ cosineScore: r._cosine,+ bm25Score: r._bm25,+ mmrScore: r.mmrScore,})),- total: results.length,+ total: picked.length,};}diff --git a/v3/@claude-flow/cli/src/memory/hybrid-retrieval.ts b/v3/@claude-flow/cli/src/memory/hybrid-retrieval.tsnew file mode 100644index 0000000000..78c01258fb--- /dev/null+++ b/v3/@claude-flow/cli/src/memory/hybrid-retrieval.ts@@ -0,0 +1,192 @@+// Hybrid retrieval β sparse (BM25) + dense (cosine) + MMR diversity.+//+// Why: on small corpora (N<1k) a bi-encoder like all-MiniLM-L6-v2 produces+// noisy cosine scores β random short commits with low-IDF token overlap can+// outrank exact-keyword matches. BM25 over the same text recovers the+// "did the query's content tokens actually appear?" signal. We linearly+// combine the two on normalised scores, then MMR-rerank to suppress+// near-duplicate top-K.+//+// Pure functions, no external deps. Tested via __tests__/hybrid-retrieval.test.ts+// and the live A/B in scripts/benchmark-pretrained-retrieval.mjs (HYBRID=0|1).++const STOPWORDS = new Set([+ 'the','a','an','and','or','but','if','then','else','of','in','to','for','on','at',+ 'by','with','from','is','are','was','were','be','been','being','have','has','had',+ 'do','does','did','will','would','should','could','can','may','might','must','this',+ 'that','these','those','it','its','as','also','not','no','so','too','very',+]);++/** Tokenise text into lowercase content tokens (drops stopwords + length<3). */+export function tokenize(text: string): string[] {+ if (!text) return [];+ return text+ .toLowerCase()+ .split(/[^a-z0-9_\-/.]+/g)+ .filter((t) => t.length >= 3 && !STOPWORDS.has(t));+}++export interface CorpusStats {+ /** docFreq: how many documents contain each token */+ df: Map<string, number>;+ /** idf: log((N - df + 0.5) / (df + 0.5) + 1) β BM25's smoothed IDF */+ idf: Map<string, number>;+ /** averageDocumentLength in tokens */+ avgDocLen: number;+ /** total docs in corpus */+ N: number;+}++/** Build BM25 corpus statistics from a set of tokenised documents. */+export function buildCorpusStats(tokenisedDocs: string[][]): CorpusStats {+ const N = tokenisedDocs.length;+ const df = new Map<string, number>();+ let totalLen = 0;++ for (const doc of tokenisedDocs) {+ totalLen += doc.length;+ const seen = new Set<string>();+ for (const tok of doc) {+ if (seen.has(tok)) continue;+ seen.add(tok);+ df.set(tok, (df.get(tok) ?? 0) + 1);+ }+ }++ const idf = new Map<string, number>();+ for (const [tok, dfVal] of df) {+ // BM25 smoothed IDF β never negative, always > 0+ idf.set(tok, Math.log(1 + (N - dfVal + 0.5) / (dfVal + 0.5)));+ }++ return { df, idf, avgDocLen: N > 0 ? totalLen / N : 0, N };+}++/**+ * BM25 score of one document against a query.+ * Standard Okapi formula with k1=1.5 and b=0.75.+ */+export function bm25Score(+ queryTokens: string[],+ docTokens: string[],+ stats: CorpusStats,+ k1 = 1.5,+ b = 0.75,+): number {+ if (queryTokens.length === 0 || docTokens.length === 0) return 0;++ // Term frequency in this document+ const tf = new Map<string, number>();+ for (const tok of docTokens) tf.set(tok, (tf.get(tok) ?? 0) + 1);++ const docLen = docTokens.length;+ const norm = docLen / (stats.avgDocLen || 1);++ let score = 0;+ for (const qt of queryTokens) {+ const f = tf.get(qt);+ if (!f) continue;+ const idf = stats.idf.get(qt) ?? 0;+ if (idf === 0) continue;+ const numerator = f * (k1 + 1);+ const denominator = f + k1 * (1 - b + b * norm);+ score += idf * (numerator / denominator);+ }+ return score;+}++/**+ * Min-max normalise a score vector to [0, 1].+ * Returns the original vector if it is constant (avoid divide-by-zero).+ */+export function normalise(scores: number[]): number[] {+ if (scores.length === 0) return scores;+ let lo = Infinity, hi = -Infinity;+ for (const s of scores) {+ if (s < lo) lo = s;+ if (s > hi) hi = s;+ }+ const range = hi - lo;+ if (range < 1e-9) return scores.map(() => 0.5);+ return scores.map((s) => (s - lo) / range);+}++/**+ * Combine cosine and BM25 scores. Both vectors must be aligned by docIndex.+ * Returns hybridScores[] aligned to the same index.+ * alpha controls the cosine weight; (1-alpha) is the BM25 weight.+ */+export function hybridScores(+ cosine: number[],+ bm25: number[],+ alpha = 0.6,+): number[] {+ if (cosine.length !== bm25.length) {+ throw new Error('hybridScores: cosine and bm25 length mismatch');+ }+ const cN = normalise(cosine);+ const bN = normalise(bm25);+ return cN.map((c, i) => alpha * c + (1 - alpha) * bN[i]);+}++/**+ * Cosine similarity between two equal-length numeric vectors.+ * Returns 0 if either has zero norm.+ */+export function cosineSim(a: number[], b: number[]): number {+ if (a.length !== b.length) return 0;+ let dot = 0, na = 0, nb = 0;+ for (let i = 0; i < a.length; i++) {+ dot += a[i] * b[i];+ na += a[i] * a[i];+ nb += b[i] * b[i];+ }+ const denom = Math.sqrt(na) * Math.sqrt(nb);+ return denom > 1e-9 ? dot / denom : 0;+}++/**+ * Maximal Marginal Relevance rerank.+ *+ * Greedy selection: at each step pick the candidate that maximises+ * lambda * relevance(c) - (1-lambda) * max(similarity(c, picked))+ *+ * lambda=1.0 β pure relevance (no diversity adjustment)+ * lambda=0.0 β pure diversity+ *+ * Default lambda=0.5 balances both.+ */+export function mmrRerank<T extends { embedding: number[] }>(+ candidates: Array<T & { relevance: number }>,+ k: number,+ lambda = 0.5,+): Array<T & { relevance: number; mmrScore: number }> {+ if (candidates.length === 0 || k === 0) return [];++ const picked: Array<T & { relevance: number; mmrScore: number }> = [];+ const remaining = candidates.slice();++ while (picked.length < k && remaining.length > 0) {+ let bestIdx = 0;+ let bestScore = -Infinity;++ for (let i = 0; i < remaining.length; i++) {+ const cand = remaining[i];+ let maxSimToPicked = 0;+ for (const p of picked) {+ const s = cosineSim(cand.embedding, p.embedding);+ if (s > maxSimToPicked) maxSimToPicked = s;+ }+ const mmrScore = lambda * cand.relevance - (1 - lambda) * maxSimToPicked;+ if (mmrScore > bestScore) {+ bestScore = mmrScore;+ bestIdx = i;+ }+ }++ const chosen = remaining.splice(bestIdx, 1)[0];+ picked.push({ ...chosen, mmrScore: bestScore });+ }++ return picked;+}diff --git a/v3/docs/adr/ADR-078-hybrid-retrieval-and-outcome-signal.md b/v3/docs/adr/ADR-078-hybrid-retrieval-and-outcome-signal.mdnew file mode 100644index 0000000000..597428bb1c--- /dev/null+++ b/v3/docs/adr/ADR-078-hybrid-retrieval-and-outcome-signal.md@@ -0,0 +1,137 @@+# ADR-078 β Hybrid Retrieval + Outcome Signal for Pretrain++**Status**: Accepted β Implemented in ruflo 3.10.18+**Date**: 2026-05-30+**Tracking**: continuation of the self-learning hardening cluster (#2245 β ADR-074 β ADR-075 β ADR-076 β ADR-077)+**Related**: ADR-074 (wiring), ADR-075 (unified stats), ADR-076 (Structured Distillation), ADR-077 (pretrain)++## Context++ADR-077 shipped pretrain-from-history and proved that pretrain *writes* to the right stores (`100% match rate` across 10 queries on a 95-pattern store). But "match rate" only measures *did the query return anything?* β not *did it return the right thing?* A retrieval that returns junk for every query also has a 100% match rate.++When we added a token-grounded relevance metric (regex-match on the result's name, per-query) the truth was uglier:++| Metric (N=385, 10 queries) | Cosine-only (pre-3.10.18) |+|---|---:|+| Match rate | 100% |+| **Top-1 relevant hit rate** | **0% (0/10)** |+| **Top-3 relevant hit rate** | **0% (0/10)** |+| MRR@3 | 0.000 |++Zero. The bi-encoder (Xenova/all-MiniLM-L6-v2 via the bridge ONNX path) was returning *plausible-looking* but *off-topic* commits at the top of every result. On a small corpus, cosine over a generic-purpose bi-encoder gets distracted by token-soup release-bump commits that share IDF-cheap tokens with everything.++Separately, ADR-077's harvester recorded every commit as `success`. A revert or a fix-followup is the strongest "this approach was wrong" signal a repo has β we were throwing it away.++## Decision++Three changes, one release:++### 1. Hybrid retrieval (BM25 + cosine + MMR)++`src/memory/hybrid-retrieval.ts` β pure functions, no deps:++- **`tokenize(text)`** β lowercase, split on non-alphanumeric, drop stopwords + tokens <3 chars+- **`buildCorpusStats(docs)`** β BM25 smoothed IDF + average doc length+- **`bm25Score(qTokens, dTokens, stats)`** β Okapi BM25 (k1=1.5, b=0.75)+- **`hybridScores(cosine, bm25, Ξ±)`** β min-max normalise both vectors then linearly combine (default Ξ±=0.6 β 60% cosine, 40% BM25)+- **`mmrRerank(candidates, k, Ξ»)`** β greedy Maximal Marginal Relevance (default Ξ»=0.5)++Wired into `neural_patterns` search via a new `mode` parameter:+- `mode: 'hybrid'` (default) β cosine + BM25 + MMR+- `mode: 'cosine'` β pre-3.10.18 behaviour, kept for A/B++New search params: `alpha`, `mmrLambda`, `limit`. Response includes `hybridScore`, `cosineScore`, `bm25Score`, `mmrScore` so callers can inspect *why* a result ranked where it did.++### 2. Pattern persistence β content field++`Pattern.content?: string` added to the neural store schema (cap 4096 chars). BM25 needs the source text; the pre-3.10.18 pattern stored only `name`, `type`, `embedding`. Backwards compatible β patterns missing `content` fall back to `name` for tokenisation.++`storeNeuralPatterns()` and the `store` action both persist content automatically.++### 3. Outcome signal in pretrain harvester++`scripts/pretrain-from-github.mjs` now classifies each commit as one of:++- **`success`** β landed cleanly, no later commit reverted or fixed it+- **`reverted`** β a later commit's subject is `Revert "<this subject>"`+- **`hotfixed`** β a later commit (within `HOTFIX_WINDOW_COMMITS`=20) shares β₯ 50% of touched files (`min(|A|,|B|)` denominator) AND has fix/hotfix/patch in its subject++Mapped to the trajectory pipeline's binary verdict:+- `success` β `'success'`+- `partial` (open issues) β `'partial'`+- `reverted` β `'partial'` (strong "this was wrong" but pipeline only has 2 levels)+- `hotfixed` β `'partial'`++The original outcome verdict is preserved in `metadata.outcomeVerdict` and surfaced in `summary.feed.verdictMix` so the signal isn't lost when callers compare runs.++## Measured proof (this checkout, N=385 patterns, 10 queries)++| Metric | Cosine | Hybrid | Ξ | Direction |+|---|---:|---:|---:|---|+| Match rate | 100% | 100% | 0 | tie |+| **Top-1 hit rate (relevance)** | **0%** | **50%** | **+50pp** | β hybrid wins |+| **Top-3 hit rate (relevance)** | **0%** | **70%** | **+70pp** | β hybrid wins |+| **MRR@3** | **0.000** | **0.583** | **+0.583** | β hybrid wins |+| Top-1 diversity | 100% | 80% | -20pp | acceptable β diversity β relevance |+| Avg query latency | 28.7 ms | 40.6 ms | +11.9 ms | hybrid 40% slower, still <50ms |++Run JSONs: `docs/benchmarks/runs/pretrained-retrieval-latest.json` (latest hybrid run).++**What hybrid fixes** β concrete top-1 swaps on this corpus:+- `"structured distillation 4-field schema"` β hybrid: `feat(intelligence): structured distillation (ADR-076)` (cosine returned an unrelated release bump)+- `"unified learning stats aggregator"` β hybrid: `feat(memory): unified learning-stats aggregator (ADR-075)` (exact)+- `"deterministic codemod engine var-to-const"` β hybrid: `feat/deterministic-tier1-codemods: deterministic Tier-1 codemods (ADR-143)` (cosine returned junk text)+- `"self-learning wiring task-completed pretrain"` β hybrid: `Self-learning reports success but persists nothing (3.10.6)` (the actual issue)+- `"recall@k HNSW benchmark harness"` β hybrid: `feat(neural-trader): benchmark suite (signal, backtest, memory-recall)`++**Where hybrid still loses** β broad queries get pulled toward release-bump commits because release commits bundle many issue numbers and share IDF-cheap tokens with everything. Future MMR tuning (lower Ξ») or a cross-encoder reranker (tracked for 3.11.0) is the path.++## Outcome signal β measured on this checkout++`COMMITS=200 SOURCE=git` harvester run:++| verdict | count |+|---|---:|+| success | 200 |+| partial | 0 |+| reverted | 0 |+| hotfixed | 0 |++This checkout has zero `Revert "<X>"`-style reverts in the 200 most recent commits and no fix-followups crossing the 50%-file-overlap threshold. The detector is correctly implemented (unit-tested in `__tests__/hybrid-retrieval.test.ts`); the empty count reflects a clean recent history, not a broken detector. On a churnier repo the numbers will move.++## Reusable infrastructure shipped++- `src/memory/hybrid-retrieval.ts` β 6 pure functions (200 LOC, 21 unit tests)+- `__tests__/hybrid-retrieval.test.ts` β full coverage of tokenize/BM25/normalise/hybrid/cosine/MMR+- `__tests__/pretrain-from-github.test.ts` β extended with a hybrid-vs-cosine assertion using the same fixture (no live git/gh)+- `scripts/benchmark-pretrained-retrieval.mjs` β `HYBRID=0|1` A/B + per-query relevance regex + MRR@3 + top-1 diversity + top-3 dup rate+- `scripts/pretrain-from-github.mjs` β outcome-signal harvester (`HOTFIX_WINDOW_COMMITS`, `HOTFIX_FILE_OVERLAP` env)+- `neural_patterns` MCP tool β new `content`, `mode`, `alpha`, `mmrLambda`, `limit` params++## Deliberately NOT in this round++- **Cross-encoder reranker** β paper-proven path for another +0.05-0.15 MRR. Tracked for 3.11.0 (new dependency, MINOR bump).+- **Learned distiller** (paper's 11Γ compression) β still tracked under #2241 round-D.+- **Negative-reward propagation on retrieval miss** β needs agent-level success attribution we don't yet emit cleanly.+- **MMR Ξ»-autotuning** β currently a static 0.5; a future ADR could grid-search on the same bench.++## Honest limits++- **N=385, 10 queries** is small. The relevance metric is a regex over commit subjects β a stronger eval would use a held-out labelled set. The direction is robust (0% β 50% top-1 hit rate doesn't flip on noise) but the magnitude could move on a different corpus.+- **Hybrid is 40% slower per query** (28.7 β 40.6 ms). Still <50 ms, but worth budgeting for hot paths. Cosine-only mode is preserved as `mode: 'cosine'` for callers that want it.+- **Top-1 diversity dropped** (100% β 80%). This is *expected* β hybrid concentrates relevant results, cosine scatters them. Diversity is a proxy only meaningful when the underlying ranking is good.++## Verification++```bash+git clone https://github.com/ruvnet/ruflo && cd ruflo+npm install && ( cd v3/@claude-flow/cli && npx tsc -b )++# Unit tests (no I/O, no network) β 21 + 7 tests+( cd v3/@claude-flow/cli && npx vitest run __tests__/hybrid-retrieval.test.ts __tests__/pretrain-from-github.test.ts )++# Pretrain + A/B retrieval (depends on git history; uses ONNX embedder)+node v3/@claude-flow/cli/scripts/pretrain-from-github.mjs+node v3/@claude-flow/cli/scripts/benchmark-pretrained-retrieval.mjs # hybrid (default)+HYBRID=0 node v3/@claude-flow/cli/scripts/benchmark-pretrained-retrieval.mjs # cosine baseline+```