π The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated
TypeScript agentic-aiagentic-frameworkagentic-workflowagentsai-agentsai-assistantai-skillsautonomous-agentsclaudecodexdsh-pluginharnessmcp-servermulti-agentmulti-agent-systemsnpmskillsswarmswarm-intelligencetypescript
fix+perf+docs(intelligence): self-learning audit, hardening fixes, honest perf numbers (3.10.7) (#2228)
rUv08b3623759da
diff --git a/CLAUDE.md b/CLAUDE.mdindex 4805697dbd..6c219809b2 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -340,7 +340,7 @@ This project is configured with Claude Flow V3 (Anti-Drift Defaults):- **Strategy**: specialized (clear roles, no overlap)- **Consensus**: raft (leader maintains authoritative state)- **Memory Backend**: hybrid (SQLite + AgentDB)-- **HNSW Indexing**: Enabled (150x-12,500x faster)+- **HNSW Indexing**: Enabled (measured ~1.9x at N=20k, ~3.2xβ4.7x at N=5k vs brute force; ANN wins above the crossover)- **Neural Learning**: Enabled (SONA)## V3 CLI Commands (26 Commands, 140+ Subcommands)@@ -352,7 +352,7 @@ This project is configured with Claude Flow V3 (Anti-Drift Defaults):| `init` | 4 | Project initialization with wizard, presets, skills, hooks || `agent` | 8 | Agent lifecycle (spawn, list, status, stop, metrics, pool, health, logs) || `swarm` | 6 | Multi-agent swarm coordination and orchestration |-| `memory` | 11 | AgentDB memory with vector search (150x-12,500x faster) |+| `memory` | 11 | AgentDB memory with HNSW vector search (measured ~1.9xβ4.7x vs brute force above crossover) || `mcp` | 9 | MCP server management and tool execution || `task` | 6 | Task creation, assignment, and lifecycle || `session` | 7 | Session state management and persistence |@@ -374,7 +374,7 @@ This project is configured with Claude Flow V3 (Anti-Drift Defaults):| `providers` | 5 | AI providers (list, add, remove, test, configure) || `plugins` | 5 | Plugin management (list, install, uninstall, enable, disable) || `deployment` | 5 | Deployment management (deploy, rollback, status, environments, release) |-| `embeddings` | 4 | Vector embeddings (embed, batch, search, init) - 75x faster with agentic-flow |+| `embeddings` | 4 | Vector embeddings (embed, batch, search, init) β agentic-flow ONNX backend (speedup unverified, no benchmark) || `claims` | 4 | Claims-based authorization (check, grant, revoke, list) || `migrate` | 5 | V2 to V3 migration with rollback support || `process` | 4 | Background process management |@@ -759,12 +759,12 @@ npx claude-flow@v3alpha hooks worker status## Intelligence System (RuVector)-V3 includes the RuVector Intelligence System:-- **SONA**: Self-Optimizing Neural Architecture (<0.05ms adaptation)-- **MoE**: Mixture of Experts for specialized routing-- **HNSW**: 150x-12,500x faster pattern search+V3 includes the RuVector Intelligence System (measured numbers: see [audit](docs/reviews/intelligence-system-audit-2026-05-29.md) + [`scripts/benchmark-intelligence.mjs`](scripts/benchmark-intelligence.mjs)):+- **SONA**: Self-Optimizing Neural Architecture (measured 0.0043ms/adapt, target <0.05ms met)+- **MoE**: Mixture of Experts for specialized routing (gate converges β confidence 0.13β0.88 after rewards)+- **HNSW**: measured ~1.9x at N=20k, ~3.2xβ4.7x at N=5k vs brute force (recall@10 ~0.99); ANN wins above the crossover, ruvector NAPI backend (WASM not active on test host)- **EWC++**: Elastic Weight Consolidation (prevents forgetting)-- **Flash Attention**: 2.49x-7.47x speedup+- **Flash Attention**: unverified β no benchmark exists for this claimThe 4-step intelligence pipeline:1. **RETRIEVE** β Fetch relevant patterns via HNSW@@ -779,7 +779,7 @@ Features:- **Document chunking**: Configurable overlap and size- **Normalization**: L2, L1, min-max, z-score- **Hyperbolic embeddings**: Poincare ball model for hierarchical data-- **75x faster**: With agentic-flow ONNX integration+- **agentic-flow ONNX integration**: speedup unverified (no benchmark; backend reported `onnx`, model all-MiniLM-L6-v2, 384-dim)- **Neural substrate**: Integration with RuVector## Hive-Mind Consensus@@ -799,15 +799,18 @@ Features:## V3 Performance Targets-| Metric | Target | Status |-|--------|--------|--------|-| HNSW Search | 150x-12,500x faster | **Implemented** (persistent) |-| Memory Reduction | 50-75% with quantization | **Implemented** (3.92x Int8) |-| SONA Integration | Pattern learning | **Implemented** (ReasoningBank) |-| Flash Attention | 2.49x-7.47x speedup | In progress |-| MCP Response | <100ms | Achieved |-| CLI Startup | <500ms | Achieved |-| SONA Adaptation | <0.05ms | In progress |+> Source of truth: [`docs/reviews/intelligence-system-audit-2026-05-29.md`](docs/reviews/intelligence-system-audit-2026-05-29.md) + [`scripts/benchmark-intelligence.mjs`](scripts/benchmark-intelligence.mjs). Numbers below are measured unless marked "target/unverified".++| Metric | Measured / Target | Status |+|--------|-------------------|--------|+| HNSW Search | ~1.9x at N=20k, ~3.2xβ4.7x at N=5k vs brute force (recall@10 ~0.99); ties/loses below crossover | **Measured** (ruvector NAPI; 150x-12,500x NOT reproduced β was brute-force fallback) |+| Int8 Quantization | 3.84x compression, reconstruction cosine 0.99999 | **Measured** |+| RaBitQ Quantization | 32x compression, 0.60ms/query (14,760-vec index) | **Measured** |+| SONA Adaptation | 0.0043ms/adapt (target <0.05ms met) | **Measured** |+| MoE Gate | converges β confidence 0.13β0.88, Q 0β99.8 after rewards | **Measured** |+| Flash Attention | 2.49x-7.47x | **Unverified** (no benchmark exists) |+| MCP Response | <100ms | target |+| CLI Startup | <500ms | target |## Environment Variablesdiff --git a/README.md b/README.mdindex d1e6974743..f77136cc3e 100644--- a/README.md+++ b/README.md@@ -202,7 +202,7 @@ claude mcp add ruflo -- npx ruflo@latest mcp start| π‘ **Comms Layer** | Zero-trust federation β agents across machines/orgs discover, authenticate, and exchange work securely || π **Swarm Coordination** | Hierarchical, mesh, and adaptive topologies with consensus || π§ **Self-Learning** | SONA neural patterns, ReasoningBank, trajectory learning |-| πΎ **Vector Memory** | HNSW-indexed AgentDB with 150x-12,500x faster search |+| πΎ **Vector Memory** | HNSW-indexed AgentDB β measured ~1.9x faster at N=20k, ~3.2xβ4.7x at N=5k vs brute force (recall@10 ~0.99); ANN wins above the crossover, ties/loses at small N. See [audit](docs/reviews/intelligence-system-audit-2026-05-29.md) + [`scripts/benchmark-intelligence.mjs`](scripts/benchmark-intelligence.mjs) || β‘ **Background Workers** | 12 auto-triggered workers (audit, optimize, testgaps, etc.) || π§© **Plugin Marketplace** | 32 native Claude Code plugins + 21 npm plugins || π **Multi-Provider** | Claude, GPT, Gemini, Cohere, Ollama with smart routing |@@ -228,7 +228,7 @@ claude mcp add ruflo -- npx ruflo@latest mcp start| π οΈ | **~210 tools, ready to call** | 5 server groups (Core, Intelligence, Agents, Memory, DevTools) plus an 18-tool gallery that runs entirely in your browser β works offline. || π | **Bring your own MCP servers** | Click the **MCP (n)** pill in the chat input β *Add Server* and paste any MCP endpoint (HTTP, SSE, or stdio). Your tools join RuFlo's native ones in the same parallel-execution flow. Run a local MCP server on `localhost:3000` and it just works. || β‘ | **Tools run in parallel** | One model response can fire 4β6+ tools at the same time. The UI shows them as cards with a *Step 1 β 2 tools completed* badge so you can see exactly what ran. |-| πΎ | **Memory that sticks** | Say *"remember my favorite color is indigo"* and ask weeks later β RuFlo recalls it. Backed by AgentDB + HNSW vector search (β₯150Γ faster than brute force). |+| πΎ | **Memory that sticks** | Say *"remember my favorite color is indigo"* and ask weeks later β RuFlo recalls it. Backed by AgentDB + HNSW vector search (measured ~1.9xβ4.7x faster than brute force above the crossover, recall@10 ~0.99). || π | **Built-in capabilities tour** | Click the question-mark icon in the sidebar β a "RuFlo Capabilities" modal opens with the full tool list, model strengths, architecture, and keyboard shortcuts. || π | **Self-hostable** | Web UI is shipped as Docker (`ruflo/src/ruvocal/Dockerfile`) with embedded Mongo. Deploy to your own Cloud Run / Fly / Kubernetes / docker-compose. The hosted [flo.ruv.io](https://flo.ruv.io/) demo is one option; running your own is fully supported. || π | **Zero install to try** | Open the hosted URL, pick a model, type a question. That's the whole onboarding. |diff --git a/docs/reviews/intelligence-system-audit-2026-05-29.md b/docs/reviews/intelligence-system-audit-2026-05-29.mdnew file mode 100644index 0000000000..08d7ded9a8--- /dev/null+++ b/docs/reviews/intelligence-system-audit-2026-05-29.md@@ -0,0 +1,89 @@+# Intelligence / Self-Learning System β Empirical Capability Audit++**Date:** 2026-05-29 Β· **Version audited:** `@claude-flow/[email protected]` (built `dist/`) Β· **Host:** darwin-arm64, Node 22+**Method:** 6 parallel auditors ran **real measurements** against the built `dist` exports, CLI, and MCP tool handlers β not documentation. Every claim is graded by evidence. Throwaway scripts were deleted; no source was modified during the audit.++> **Honesty mandate.** This audit was commissioned to *measure, benchmark, and confirm* β including confirming where claims do **not** hold. Headline performance multipliers in `CLAUDE.md` are largely hardcoded doc strings with no benchmark behind them; several are unsubstantiated and one is fabricated at runtime. At the same time, the **core self-learning loop is genuinely real and was measured end-to-end.** Both halves of that are reported plainly below.++---++## TL;DR++- **The learning loop is real.** A success/failure verdict on a recorded trajectory measurably and directionally changes stored pattern confidence, persisted to disk, surviving across separate processes. Q-learning routing feedback genuinely steers the next routing decision cross-process. This is not theater.+- **The big "Γ" numbers are not.** "HNSW 150xβ12,500x" measured **1.48Γ peak**; "Flash Attention 2.49β7.47Γ" is **fabricated with `Math.random()` at runtime**; "75Γ embeddings" and "RaBitQ 2.70Γ retrieval" have **no benchmark** anywhere.+- **Confirmed-real with honest numbers:** Int8 **3.92Γ** memory, RaBitQ **32Γ** memory, MoE 8-expert gating (measured convergence), SONA WASM adapt **0.0042 ms** (beats the <0.05 ms claim).+- **One critical correctness bug:** the CLI **silently inverts negative reward** (`route feedback -r -1.0` records **+1.00**) β a user training against a bad agent the documented way *reinforces* it.++---++## Capability matrix++### β CONFIRMED-REAL (measured)++| Capability | Evidence (measured) |+|---|---|+| **4-step learning loop** RETRIEVEβJUDGEβDISTILL | Success verdict pushed pattern confidence **0.906β1.0**; subsequent failure pulled **1.0β0.952**; counters persist across separate processes. Steps share one `LocalReasoningBank`+`SonaCoordinator` and feed each other. |+| **ReasoningBank file persistence** | Cross-process: stored in proc 1 β reloaded from disk in proc 2. `.claude-flow/neural/patterns.json` (+`stats.json`). |+| **Pattern storeβsearch roundtrip** (#2226 fix) | Holds on both direct and MCP paths (`controller:"bridge-store", impl:"real-hnsw-indexed"`). 3.10.6 fix verified end-to-end. |+| **Memory bridge** (import_claude / bridge_status / search_unified) | Real import (14 entries, 3 projects), results carry `source:"claude-code"` attribution. Persists to `.swarm/memory.db` ns `claude-memories`. |+| **MoE β 8 experts + gating** | Genuine 384β128β8 softmax MLP + REINFORCE backprop. Measured: coder expert probability **0.081β0.994** after 200 rewards. |+| **SONA <0.05 ms adaptation** (WASM) | `SonaInstantWasm.instantAdapt` measured **0.00417 ms/call** (200k warmed) β 12Γ under the claim. EMA + adaptive-rank schedule. |+| **Q-learning self-improvement** (mechanism) | Q-table is read at inference and argmax'd (no static fallback). Cross-process: penalize `architect` β router switches greedy pick to `researcher`, persisted to `.swarm/q-learning-model.json`. |+| **Int8 quantization 3.92Γ** | Measured **3.918Γ** (1536β392 bytes), reconstruction cosine **0.99999**. |+| **RaBitQ 32Γ memory** | Real WASM (`@ruvector/[email protected]`), builds a real 1-bit index, compressionRatio **32**. Not a stub. |+| **3-tier model routing** (hybrid) | "typo"βsonnet/11%, "architect distributed consensus"βopus/60%. Static keyword complexity + a **real persisted Beta (Thompson) bandit** that measurably shifts model choice after `model-outcome` feedback. |+| **MCP trajectory start/end** | Real lifecycle; `end` triggers SONA learning (pattern @ 55% confidence), persists to `.swarm/memory.db` + `.swarm/sona-patterns.json`, cross-process verified. |++### π‘ PARTIAL (real core, overstated or with gaps)++| Capability | What's real / what's not |+|---|---|+| **EWC++** | Penalty math `Β½Β·Ξ£ Fα΅’(ΞΈα΅’βΞΈ*α΅’)Β²` is correctly implemented and runs. **But "Fisher information" is a heuristic proxy** β `Fα΅’ = \|wα΅’\|Β·Ξ»` (ruvllm) / `embeddingα΅’Β²` (TS), not gradient curvature E[gΒ²]. `forgettingRate = 1βe^(βtasksΒ·0.1)` is a label, not a measurement. |+| **MicroLoRA** | JS `LoraAdapter` forward/backward is real low-rank math. **The WASM adapter the MCP tools actually call is inert** β output L1 delta **0.000000** after 5000 adapts (B stays zero; gradients accumulate but never flush). |+| **CONSOLIDATE (MCP path)** | Real `EWCConsolidator`, but MCP `trajectory-end` feeds it a **synthetic `Math.sin()` gradient** (`hooks-tools.js:2481`), not the embedding-derived one the library path uses. Fisher file didn't persist in a clean run. |+| **ONNX embeddings** | 384-dim shape correct, but on this host `sharp` native build fails β **silent fallback to mock embeddings still labeled `Xenova/all-MiniLM-L6-v2`**. Synonyms scored **β0.988**, unrelated text **+0.775**. Operator cannot tell mock from real β an observability defect regardless of environment. |+| **Trajectory tracking** | MCP tools real & persist; **CLI `hooks intelligence trajectory-*` subcommands are no-op stubs** (positional arg ignored β always render the status dashboard). MCP `step` is **in-memory only** between start/end (lost if server restarts mid-trajectory). |+| **neural train** | Real epoch loop with contrastive loss + LoRA backend (JS fallback does real rank-2 algebra). **But emits zero output in non-TTY** β work happens invisibly. |++### β UNSUBSTANTIATED / FABRICATED++| Claim | Reality |+|---|---|+| **HNSW "150xβ12,500x faster"** | Measured **peak 1.48Γ at N=20k**; *slower* than brute force below Nβ5k. The multipliers are hardcoded doc strings; the benchmark command's "recall" is a hardcoded `0.99` constant. Baseline undefined. (A real ANN index exists β only the *magnitude* and missing baseline are the problem; 150Γ would need millions of vectors.) |+| **Flash Attention "2.49xβ7.47x"** | **Fabricated at runtime:** `attention-coordinator.ts:972` β `flashSpeedup = 2.49 + Math.random()*4.98`. A correct tiled kernel exists (`blockAttention`, RMSE 2.6e-8) but measures only **~1.1Γ**; the *default* path is a lossy top-K sparse approximation (12% of keys, RMSE 0.17) measuring **0.77Γβ3.62Γ**, never 7.47Γ. |+| **RaBitQ "2.70x retrieval"** | Not measured anywhere in source. (The 32Γ *memory* claim is confirmed; the retrieval-speed number is not.) |+| **Embeddings "75x faster"** | Doc literal, no baseline, never measured. |++---++## Bugs & defects found (prioritized)++| # | Sev | Defect | Location |+|---|-----|--------|----------|+| 1 | π΄ **Critical** | **CLI inverts negative reward.** `route feedback -r -1.0` (and `--reward -1.0`) parses as **+1.00** β negative feedback reinforces the bad agent. Only `--reward=-1.0` (equals form) preserves the sign; the command's own help example is broken. | `src/commands/route.ts` flag parser |+| 2 | π High | **Fabricated speedup metric** reported as real telemetry via `Math.random()`. | `v3/@claude-flow/swarm/src/attention-coordinator.ts:972` |+| 3 | π High | **Silent mock-embedding fallback mislabeled as the real ONNX model** β no way to distinguish mock from real output. | `memory-initializer.ts` embedding path; agentdb `EmbeddingService` |+| 4 | π‘ Med | **`hooks_intelligence_learn` is cosmetic** β reads/echoes stats; does not run a learning cycle despite its name. | `hooks-tools.js` (~:2920) |+| 5 | π‘ Med | **MCP `trajectory-end` consolidation uses a synthetic `Math.sin()` gradient**, not the trajectory's real embeddings. | `hooks-tools.js:2481` |+| 6 | π‘ Med | **CLI `hooks intelligence trajectory-*` subcommands are no-op stubs** (render the status dashboard). | `src/commands/hooks.js:1758` |+| 7 | π‘ Med | **WASM MicroLoRA `apply()` is inert** (B never flushed β output unchanged). | `@ruvector/ruvllm-wasm`; `ruvllm-wasm.ts` |+| 8 | π’ Low | **Benchmark "recall" is a hardcoded `0.99`** constant, not measured. | `commands/ruvector/benchmark.js:377` |+| 9 | π’ Low | **`neural train` / `hooks intelligence` emit nothing in non-TTY**; documented `node dist/src/index.js` entry prints nothing (use `bin/cli.js`). | `commands/neural.js`, `commands/hooks.js` |+| 10 | π’ Low | Q-router: stale route cache hides learning until 50 updates; `--explore false` is ignored; Beta bandit priors are global-per-model, not per-task. | `q-learning-router.ts`, `route.ts`, `model-router.ts` |++---++## Honest bottom line++The self-learning system is **substantially real where it counts**: there is a genuine closed loop in which task outcomes update persisted pattern confidence and routing Q-values, and those persisted values demonstrably change subsequent behavior across processes. MoE gating, SONA WASM latency, Int8/RaBitQ compression, and the memory bridge all hold up under direct measurement. This is a real reinforcement-style memory system, not a faΓ§ade.++What does **not** hold up is the **performance-multiplier marketing**: the HNSW and Flash-Attention speedups are unsubstantiated (and one is literally randomized at runtime), and the headline embedding/quantization "ΓN faster" figures have no benchmark behind them. Several capabilities are real algorithms that are **implemented but inert or disconnected** (WASM MicroLoRA, MCP consolidation gradient, the `learn` tool, CLI trajectory subcommands). And one genuine **correctness bug** (negative-reward inversion) means the documented self-improvement path can train the system *backwards*.++**Recommended next steps** (not yet applied β this branch is the audit only):+1. **Fix #1 (negative-reward inversion)** β highest priority; it actively corrupts learning. (Follow-up to #2222.)+2. **Remove the `Math.random()` speedup (#2)** β replace with a real measurement or drop the metric; it is a credibility liability.+3. **Make the mock-embedding fallback observable (#3)** β surface `embeddingBackend: "mock"|"onnx"` everywhere the model name is reported.+4. **Correct the `CLAUDE.md` / perf-table claims** to measured values (Int8 3.92Γ, RaBitQ 32Γ memory, SONA 0.004 ms confirmed; HNSW/Flash/embeddings marked "unverified / approximate" with the real numbers).+5. Wire or remove the inert pieces (#4β#7) so named capabilities are either real or not advertised.++*Per-subsystem raw evidence is preserved in the audit run; load-bearing file:line references are inline above.*diff --git a/package-lock.json b/package-lock.jsonindex 0758cb1592..77b586a168 100644--- a/package-lock.json+++ b/package-lock.json@@ -1,12 +1,12 @@{"name": "claude-flow",- "version": "3.10.6",+ "version": "3.10.7","lockfileVersion": 3,"requires": true,"packages": {"": {"name": "claude-flow",- "version": "3.10.6",+ "version": "3.10.7","license": "MIT","dependencies": {"@claude-flow/cli-core": "^3.7.0-alpha.5",diff --git a/package.json b/package.jsonindex f3e248daa6..fa2f5f7af2 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "claude-flow",- "version": "3.10.6",+ "version": "3.10.7","description": "Ruflo - Enterprise AI agent orchestration for Claude Code. Deploy 60+ specialized agents in coordinated swarms with self-learning, fault-tolerant consensus, vector memory, and MCP integration","main": "dist/index.js","type": "module",diff --git a/ruflo/README.md b/ruflo/README.mdindex 0b7eee30a0..d1e6974743 100644--- a/ruflo/README.md+++ b/ruflo/README.md@@ -1,6 +1,6 @@<div align="center">-[](https://flo.ruv.io/)+[](https://cognitum.one/agentic-engineering)[](https://flo.ruv.io/)[](https://goal.ruv.io/)@@ -374,13 +374,17 @@ User --> Claude Code / CLI## Documentation-Three docs for three audiences:+Four docs for four audiences:| Doc | When to read it ||-----|-----------------|| **[Status](docs/STATUS.md)** | See what currently works β capability counts, test baselines, recent fixes, what's next. The *is-it-ready* doc. || **[User Guide](docs/USERGUIDE.md)** | Daily reference β every command, every config flag, every plugin. The *how-do-I* doc. |+| **[Benchmarks](https://gist.github.com/ruvnet/298f8c668c8859b369f91734a0e9cbbe)** | v3.8.0 SOTA matrix vs LangGraph / AutoGen / CrewAI on darwin-arm64 + linux-x64. ruflo wins cold start, single turn, RSS by 1.3Γβ1953Γ. The *is-it-fast* doc. || **[Verification](verification.md)** | Cryptographically prove your installed bytes match the signed witness β `ruflo verify`. The *trust-but-verify* doc. |+| **[Team Gateway Checklist](docs/TEAM-GATEWAY-CHECKLIST.md)** | Before-merge gates, dual-mode handoff, memory namespace sharing, and witness manifest entry per merge. The *safer-team-workflows* doc. |++Benchmark internals (for reproduction): [`sota-workload-spec.md`](https://github.com/ruvnet/ruflo/blob/perf/sota-comparator-benchmarks/docs/benchmarks/sota-workload-spec.md) Β· [`SOTA-PROGRESS.md`](https://github.com/ruvnet/ruflo/blob/perf/sota-comparator-benchmarks/docs/benchmarks/SOTA-PROGRESS.md) Β· [raw matrix JSON: darwin](https://github.com/ruvnet/ruflo/blob/perf/sota-comparator-benchmarks/docs/benchmarks/sota-matrix.json) Β· [linux](https://github.com/ruvnet/ruflo/blob/perf/sota-comparator-benchmarks/docs/benchmarks/sota-matrix-linux.json)User Guide section index:diff --git a/ruflo/package.json b/ruflo/package.jsonindex cc8e86c847..8b581195f7 100644--- a/ruflo/package.json+++ b/ruflo/package.json@@ -1,6 +1,6 @@{"name": "ruflo",- "version": "3.10.6",+ "version": "3.10.7","description": "Ruflo - Enterprise AI agent orchestration platform. Deploy 60+ specialized agents in coordinated swarms with self-learning, fault-tolerant consensus, vector memory, and MCP integration","main": "bin/ruflo.js","type": "module",diff --git a/scripts/benchmark-intelligence.mjs b/scripts/benchmark-intelligence.mjsnew file mode 100755index 0000000000..18ba56e567--- /dev/null+++ b/scripts/benchmark-intelligence.mjs@@ -0,0 +1,635 @@+#!/usr/bin/env node+/**+ * benchmark-intelligence.mjs β Real, reusable benchmark harness for the+ * RuVector / AgentDB intelligence stack.+ *+ * Measures, on the machine it runs on, against the BUILT exports under+ * v3/@claude-flow/cli/dist/src/...+ * (never against source, never hardcoded):+ *+ * 1. HNSW search vs in-process brute-force cosine baseline+ * at N = 1000, 5000, 20000, 50000:+ * - per-query ms (HNSW + brute force)+ * - speedup ratio (brute / hnsw)+ * - recall@10 (HNSW results vs exact top-10)+ * 2. Int8 quantization: measured compression ratio + reconstruction cosine.+ * 3. RaBitQ: memory compression ratio (+ retrieval speed if a populated+ * index is feasible, else null with a reason).+ * 4. SONA WASM adapt latency (ms/call, warmed).+ * 5. MoE gate: confirm the gate LEARNS (probability/Q shift after rewards).+ * 6. Embedding backend actually in use (onnx vs mock) β honest.+ *+ * DESIGN NOTES+ * ------------+ * - All vectors are generated with a seeded deterministic RNG so the run is+ * reproducible and safe to re-run. We deliberately do NOT route HNSW/quant+ * benchmarks through the embedding backend: that backend can be mock on a+ * given machine, which would make the structural benchmarks non-deterministic+ * and conflate two separate measurements. The embedding backend is instead+ * reported HONESTLY as its own item (#6).+ * - Every number printed/emitted comes from a measurement in THIS process.+ * Unmeasurable items are emitted as `null` with a `reason` string.+ * - Exit code is 0 on success (a benchmark being unmeasurable is not a failure;+ * only an unexpected crash is). The script prints a markdown table to stdout+ * and writes a machine-readable JSON object after a `===BENCH_JSON===` marker.+ *+ * USAGE+ * node scripts/benchmark-intelligence.mjs # default sizes+ * node scripts/benchmark-intelligence.mjs --sizes 1000,5000+ * node scripts/benchmark-intelligence.mjs --queries 50 --dims 384+ * node scripts/benchmark-intelligence.mjs --json-only+ *+ * Created for the ruflo intelligence stack. Co-Authored-By: RuFlo <[email protected]>+ */++import { fileURLToPath } from 'node:url';+import path from 'node:path';++const __dirname = path.dirname(fileURLToPath(import.meta.url));+const REPO_ROOT = path.resolve(__dirname, '..');+const DIST = path.join(REPO_ROOT, 'v3', '@claude-flow', 'cli', 'dist', 'src');++// ----------------------------------------------------------------------------+// CLI args+// ----------------------------------------------------------------------------+function parseArgs(argv) {+ const args = { sizes: [1000, 5000, 20000, 50000], queries: 30, dims: 384, jsonOnly: false };+ for (let i = 2; i < argv.length; i++) {+ const a = argv[i];+ if (a === '--sizes') args.sizes = argv[++i].split(',').map((s) => parseInt(s.trim(), 10)).filter(Boolean);+ else if (a === '--queries') args.queries = parseInt(argv[++i], 10);+ else if (a === '--dims') args.dims = parseInt(argv[++i], 10);+ else if (a === '--json-only') args.jsonOnly = true;+ else if (a === '--help' || a === '-h') {+ console.log('Usage: node scripts/benchmark-intelligence.mjs [--sizes N,N] [--queries K] [--dims D] [--json-only]');+ process.exit(0);+ }+ }+ return args;+}+const ARGS = parseArgs(process.argv);++const log = (...m) => { if (!ARGS.jsonOnly) console.log(...m); };++// ----------------------------------------------------------------------------+// Deterministic RNG (mulberry32) + unit-vector generator+// ----------------------------------------------------------------------------+function mulberry32(seed) {+ let a = seed >>> 0;+ return function () {+ a |= 0; a = (a + 0x6d2b79f5) | 0;+ let t = Math.imul(a ^ (a >>> 15), 1 | a);+ t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;+ return ((t ^ (t >>> 14)) >>> 0) / 4294967296;+ };+}++/** Generate a clustered set of unit-length Float32 vectors (deterministic). */+function makeDataset(n, dims, seed) {+ const rng = mulberry32(seed);+ // A handful of cluster centroids so nearest-neighbour structure is non-trivial.+ const numClusters = Math.max(8, Math.round(Math.sqrt(n) / 4));+ const centroids = [];+ for (let c = 0; c < numClusters; c++) {+ const v = new Float32Array(dims);+ for (let d = 0; d < dims; d++) v[d] = rng() * 2 - 1;+ normalize(v);+ centroids.push(v);+ }+ const vectors = new Array(n);+ for (let i = 0; i < n; i++) {+ const base = centroids[i % numClusters];+ const v = new Float32Array(dims);+ for (let d = 0; d < dims; d++) v[d] = base[d] + (rng() * 2 - 1) * 0.35; // jitter around centroid+ normalize(v);+ vectors[i] = v;+ }+ return vectors;+}++function normalize(v) {+ let s = 0;+ for (let i = 0; i < v.length; i++) s += v[i] * v[i];+ const inv = s > 0 ? 1 / Math.sqrt(s) : 0;+ for (let i = 0; i < v.length; i++) v[i] *= inv;+ return v;+}++function cosine(a, b) {+ let dot = 0, na = 0, nb = 0;+ for (let i = 0; i < a.length; i++) { dot += a[i] * b[i]; na += a[i] * a[i]; nb += b[i] * b[i]; }+ const denom = Math.sqrt(na) * Math.sqrt(nb);+ return denom > 0 ? dot / denom : 0;+}++/** Exact top-k by cosine (brute force) β returns array of ids. */+function bruteTopK(vectors, query, k) {+ const scored = new Array(vectors.length);+ for (let i = 0; i < vectors.length; i++) scored[i] = [i, cosine(query, vectors[i])];+ scored.sort((x, y) => y[1] - x[1]);+ return scored.slice(0, k).map((s) => s[0]);+}++const now = () => performance.now();+const round = (x, d = 4) => (x == null || Number.isNaN(x) ? null : Number(x.toFixed(d)));++// ----------------------------------------------------------------------------+// 1. HNSW vs brute-force+// ----------------------------------------------------------------------------+async function benchHnsw() {+ const out = { unit: 'ms/query', backend: null, byN: {}, note: '' };+ let createVectorDB, getStatus, loadRuVector;+ try {+ ({ createVectorDB, getStatus, loadRuVector } = await import(path.join(DIST, 'ruvector', 'vector-db.js')));+ await loadRuVector();+ out.backend = getStatus();+ } catch (e) {+ out.error = `failed to load ruvector vector-db: ${e.message}`;+ return out;+ }++ // ruvector createVectorDB enforces 384 dims on this build; honour that.+ const dims = ARGS.dims;+ const K = 10;++ for (const N of ARGS.sizes) {+ const entry = { n: N };+ let dataset, queries;+ try {+ dataset = makeDataset(N, dims, 1234 + N);+ // Queries: reuse a deterministic subset perturbed slightly so they are+ // near (but not identical to) indexed points.+ const qRng = mulberry32(99 + N);+ queries = [];+ for (let q = 0; q < ARGS.queries; q++) {+ const src = dataset[Math.floor(qRng() * N)];+ const v = new Float32Array(dims);+ for (let d = 0; d < dims; d++) v[d] = src[d] + (qRng() * 2 - 1) * 0.05;+ queries.push(normalize(v));+ }+ } catch (e) {+ entry.error = `dataset build failed: ${e.message}`;+ out.byN[N] = entry;+ continue;+ }++ // --- Build HNSW index ---+ let db, buildMs;+ try {+ const t0 = now();+ db = await createVectorDB(dims);+ for (let i = 0; i < N; i++) await db.insert(dataset[i], String(i));+ buildMs = now() - t0;+ } catch (e) {+ entry.error = `hnsw build failed: ${e.message}`;+ out.byN[N] = entry;+ continue;+ }+ entry.buildMs = round(buildMs, 2);+ entry.indexSize = await db.size();++ // --- HNSW query timing + recall ---+ // Warm the index first: the native NAPI search path has per-call marshalling+ // + JIT + page-fault overhead that dominates the FIRST touch of each query.+ // The brute-force baseline below runs in an already-hot JS loop, so timing+ // HNSW cold would compare a cold path against a warm one. We discard one full+ // pass so both sides are measured at steady state β this measures search+ // performance, not first-call overhead.+ for (const q of queries) await db.search(q, K);++ let hnswTotal = 0, recallHits = 0, recallTotal = 0;+ const exactByQuery = queries.map((q) => bruteTopK(dataset, q, K)); // ground truth+ for (let qi = 0; qi < queries.length; qi++) {+ const q = queries[qi];+ const t0 = now();+ const res = await db.search(q, K);+ hnswTotal += now() - t0;+ const ids = (res || []).map((r) => parseInt(r.id, 10));+ const truth = new Set(exactByQuery[qi]);+ for (const id of ids) if (truth.has(id)) recallHits++;+ recallTotal += K;+ }+ const hnswPerQuery = hnswTotal / queries.length;++ // --- Brute-force baseline timing (independent of ground-truth precompute) ---+ let bruteTotal = 0;+ for (const q of queries) {+ const t0 = now();+ bruteTopK(dataset, q, K);+ bruteTotal += now() - t0;+ }+ const brutePerQuery = bruteTotal / queries.length;++ entry.hnswMsPerQuery = round(hnswPerQuery, 5);+ entry.bruteMsPerQuery = round(brutePerQuery, 5);+ entry.speedup = round(hnswPerQuery > 0 ? brutePerQuery / hnswPerQuery : null, 2);+ entry.recallAt10 = round(recallTotal > 0 ? recallHits / recallTotal : null, 4);+ out.byN[N] = entry;++ if (db.clear) await db.clear();+ log(` HNSW N=${N}: build=${entry.buildMs}ms hnsw=${entry.hnswMsPerQuery}ms brute=${entry.bruteMsPerQuery}ms speedup=${entry.speedup}x recall@10=${entry.recallAt10}`);+ }+ return out;+}++// ----------------------------------------------------------------------------+// 2. Int8 quantization+// ----------------------------------------------------------------------------+async function benchInt8() {+ const out = {};+ let encodeEmbedding, decodeEmbedding, encodedByteSize;+ try {+ ({ encodeEmbedding, decodeEmbedding, encodedByteSize } = await import(path.join(DIST, 'memory', 'embedding-quantization.js')));+ } catch (e) {+ return { error: `failed to load embedding-quantization: ${e.message}` };+ }+ const dims = ARGS.dims;+ const SAMPLES = 200;+ const rng = mulberry32(4242);+ let cosSum = 0;+ let rawBytes = 0, encBase64Bytes = 0, quantRawBytes = 0;+ for (let s = 0; s < SAMPLES; s++) {+ const v = new Float32Array(dims);+ for (let d = 0; d < dims; d++) v[d] = rng() * 2 - 1;+ normalize(v);+ const encoded = encodeEmbedding(v); // "inline:<base64>"+ const decoded = decodeEmbedding(encoded); // Float32Array | null+ if (!decoded) continue;+ cosSum += cosine(v, decoded);+ rawBytes += dims * 4; // float32 source bytes+ // measured base64 transport payload (the blob, sans "inline:" prefix)+ const b64 = encoded.startsWith('inline:') ? encoded.slice(7) : encoded;+ encBase64Bytes += b64.length; // chars == bytes for base64 ASCII+ // measured raw quantized byte count: decode the base64 to get its true+ // pre-encoding size (header + 1 byte/dim). We do NOT assume the format.+ quantRawBytes += Buffer.from(b64, 'base64').length;+ }+ out.dims = dims;+ out.samples = SAMPLES;+ out.reconstructionCosine = round(cosSum / SAMPLES, 6);+ out.rawBytesPerVec = round(rawBytes / SAMPLES, 1); // 1536 for 384-d f32+ out.quantizedRawBytesPerVec = round(quantRawBytes / SAMPLES, 1); // ~400 (header+int8)+ out.base64BytesPerVec = round(encBase64Bytes / SAMPLES, 1); // ~536 transport+ out.encodedByteSizeReported = encodedByteSize ? encodedByteSize(dims) : null;+ // Honest int8 compression = float32 source bytes / int8 quantized bytes.+ out.compressionRatioInt8 = round(rawBytes / quantRawBytes, 3);+ // Transport (base64) ratio β what actually lands in the embedding_ref column.+ out.compressionRatioBase64 = round(rawBytes / encBase64Bytes, 3);+ log(` Int8: reconstructionCosine=${out.reconstructionCosine} compression(int8)=${out.compressionRatioInt8}x compression(base64)=${out.compressionRatioBase64}x`);+ return out;+}++// ----------------------------------------------------------------------------+// 3. RaBitQ+// ----------------------------------------------------------------------------+async function benchRabitq() {+ const out = {};+ let buildRabitqIndex, getRabitqStatus, searchRabitq;+ try {+ ({ buildRabitqIndex, getRabitqStatus, searchRabitq } = await import(path.join(DIST, 'memory', 'rabitq-index.js')));+ } catch (e) {+ return { error: `failed to load rabitq-index: ${e.message}` };+ }+ // Memory compression ratio is a structural property: 1-bit packing of a+ // float32 vector = 32x. The WASM module reports the measured ratio at build+ // time. buildRabitqIndex pulls from a SQLite memory DB β if no populated DB+ // exists we cannot measure retrieval speed; report that honestly.+ const status = getRabitqStatus();+ out.available = !!status.available;+ out.statusBefore = status;++ let build = null;+ try {+ build = await buildRabitqIndex({ dimensions: ARGS.dims });+ } catch (e) {+ out.buildError = e.message;+ }+ out.build = build;++ if (build && build.success && build.vectorCount > 0) {+ out.compressionRatio = round(build.compressionRatio, 3);+ out.buildTimeMs = round(build.buildTimeMs, 3);+ // Retrieval timing on the populated index.+ try {+ const rng = mulberry32(7);+ const q = Array.from({ length: ARGS.dims }, () => rng() * 2 - 1);+ const reps = 20;+ const t0 = now();+ for (let i = 0; i < reps; i++) await searchRabitq(q, { k: 10 });+ out.searchMsPerQuery = round((now() - t0) / reps, 5);+ } catch (e) {+ out.searchMsPerQuery = null;+ out.searchNote = `retrieval not measured: ${e.message}`;+ }+ } else {+ // No populated SQLite vector store on this machine β compression ratio is+ // reported from status/build if the WASM module surfaced it, else from the+ // documented 1-bit packing invariant is NOT assumed; we mark it null.+ out.compressionRatio = build && typeof build.compressionRatio === 'number' && build.compressionRatio > 0+ ? round(build.compressionRatio, 3)+ : (typeof status.compressionRatio === 'number' && status.compressionRatio > 0 ? round(status.compressionRatio, 3) : null);+ out.searchMsPerQuery = null;+ out.searchNote = 'not measured: no populated RaBitQ/SQLite index available on this machine (build returned vectorCount=0)';+ }+ log(` RaBitQ: available=${out.available} compressionRatio=${out.compressionRatio} searchMsPerQuery=${out.searchMsPerQuery ?? 'null'}`);+ return out;+}++// ----------------------------------------------------------------------------+// 4. SONA WASM adapt latency (warmed)+// ----------------------------------------------------------------------------+async function benchSona() {+ const out = {};+ let isRuvllmWasmAvailable, initRuvllmWasm, createSonaInstant;+ try {+ ({ isRuvllmWasmAvailable, initRuvllmWasm, createSonaInstant } = await import(path.join(DIST, 'ruvector', 'ruvllm-wasm.js')));+ } catch (e) {+ return { error: `failed to load ruvllm-wasm: ${e.message}` };+ }+ const available = await isRuvllmWasmAvailable();+ out.wasmAvailable = available;+ if (!available) {+ out.adaptMsPerCall = null;+ out.note = 'not measured: @ruvector/ruvllm-wasm not available on this machine';+ log(' SONA: WASM not available');+ return out;+ }+ await initRuvllmWasm();+ const sona = await createSonaInstant({ hiddenDim: 64 });+ // Warm-up (JIT + WASM page faults).+ for (let i = 0; i < 1000; i++) sona.adapt(0.7 + (i % 3) * 0.1);+ // Measured loop.+ const ITER = 20000;+ const rng = mulberry32(11);+ const t0 = now();+ for (let i = 0; i < ITER; i++) sona.adapt(rng());+ const totalMs = now() - t0;+ out.iterations = ITER;+ out.totalMs = round(totalMs, 3);+ out.adaptMsPerCall = round(totalMs / ITER, 6);+ out.targetMet_0_05ms = out.adaptMsPerCall != null ? out.adaptMsPerCall < 0.05 : null;+ if (sona.reset) sona.reset();+ log(` SONA: ${out.adaptMsPerCall} ms/adapt-call (warmed, ${ITER} iters)`);+ return out;+}++// ----------------------------------------------------------------------------+// 5. MoE gate learns (Q-value / probability shift after rewards)+// ----------------------------------------------------------------------------+async function benchMoeGate() {+ const out = {};+ let createQLearningRouter;+ try {+ ({ createQLearningRouter } = await import(path.join(DIST, 'ruvector', 'q-learning-router.js')));+ } catch (e) {+ return { error: `failed to load q-learning-router: ${e.message}` };+ }+ let router;+ try {+ router = createQLearningRouter({ saveInterval: 1e9 }); // never auto-persist during bench+ } catch (e) {+ return { error: `failed to construct router: ${e.message}` };+ }++ const task = 'optimize the database query performance bottleneck';+ // Discover available actions from an initial (greedy) decision.+ const before = router.route(task, false);+ const actions = (before.alternatives && before.alternatives.length)+ ? before.alternatives.map((a) => a.route)+ : [before.route];+ if (actions.length < 2) {+ out.note = 'gate exposed fewer than 2 actions; cannot demonstrate competitive shift';+ }+ // Pick a "good" action and reward it repeatedly; pick a "bad" action and+ // penalise it. We then verify the gate's Q-value / probability for the good+ // action rose relative to before.+ const good = before.route; // initial greedy pick (argmax of qValues)+ const bad = actions.find((a) => a !== good) || good;++ // The greedy decision's chosen route is the argmax, so its Q is the max of+ // the qValues vector. We read the Q of whatever route is greedily chosen+ // before and after training. After rewarding `good`, a learning gate should+ // (a) keep choosing `good` and (b) have raised its Q for that context.+ const maxQ = (decision) => (decision.qValues && decision.qValues.length ? Math.max(...decision.qValues) : null);+ // Q assigned specifically to the `good` route: if `good` is the greedy pick+ // it equals maxQ; otherwise it's its score in alternatives.+ const qForRoute = (decision, route) => {+ if (decision.route === route) return maxQ(decision);+ const alt = (decision.alternatives || []).find((a) => a.route === route);+ return alt ? alt.score : null;+ };++ const beforeGoodQ = qForRoute(before, good);+ const beforeConf = before.confidence;++ // Train: reward `good`, penalise `bad`.+ const REWARDS = 200;+ for (let i = 0; i < REWARDS; i++) {+ router.update(task, good, 1.0, task);+ if (bad !== good) router.update(task, bad, -1.0, task);+ }++ const after = router.route(task, false); // greedy after training+ const afterGoodQ = qForRoute(after, good);+ const afterConf = after.confidence;+ out.afterGreedyRoute = after.route;+ out.goodStillChosen = after.route === good;++ out.actionsObserved = actions.length;+ out.goodAction = good;+ out.badAction = bad;+ out.rewardsApplied = REWARDS;+ out.beforeGoodQ = round(beforeGoodQ, 5);+ out.afterGoodQ = round(afterGoodQ, 5);+ out.qShift = (beforeGoodQ != null && afterGoodQ != null) ? round(afterGoodQ - beforeGoodQ, 5) : null;+ out.beforeConfidence = round(beforeConf, 5);+ out.afterConfidence = round(afterConf, 5);+ out.confidenceShift = (beforeConf != null && afterConf != null) ? round(afterConf - beforeConf, 5) : null;+ // The gate "learns" if reward changed its internal valuation in the rewarded+ // direction (Q rose) OR its confidence in the greedy pick rose.+ out.gateLearned = (out.qShift != null && out.qShift > 0) || (out.confidenceShift != null && out.confidenceShift > 0);+ const stats = router.getStats ? router.getStats() : null;+ out.routerStats = stats;+ log(` MoE gate: qShift=${out.qShift} confShift=${out.confidenceShift} learned=${out.gateLearned}`);+ return out;+}++// ----------------------------------------------------------------------------+// 6. Embedding backend honesty+// ----------------------------------------------------------------------------+async function benchEmbeddingBackend() {+ const out = {};+ let generateEmbedding;+ try {+ ({ generateEmbedding } = await import(path.join(DIST, 'memory', 'memory-initializer.js')));+ } catch (e) {+ return { error: `failed to load memory-initializer: ${e.message}` };+ }+ try {+ const r = await generateEmbedding('benchmark probe: authentication and database optimization patterns');+ out.backend = r.backend; // 'onnx' | 'mock' β the authoritative signal+ out.model = r.model;+ out.dimensions = r.dimensions;+ out.honest = `backend=${r.backend} (model string '${r.model}' is reported regardless of backend; backend field is authoritative)`;+ } catch (e) {+ out.backend = null;+ out.note = `not measured: ${e.message}`;+ }+ log(` Embedding backend: ${out.backend} (model=${out.model}, dims=${out.dimensions})`);+ return out;+}++// ----------------------------------------------------------------------------+// Markdown report+// ----------------------------------------------------------------------------+function printMarkdown(results) {+ const lines = [];+ lines.push('');+ lines.push('## Intelligence Benchmark β Measured Results');+ lines.push('');+ lines.push(`- Host: ${process.platform}/${process.arch}, Node ${process.version}`);+ lines.push(`- dist: ${path.relative(REPO_ROOT, DIST)}`);+ lines.push(`- dims=${ARGS.dims}, queries/size=${ARGS.queries}`);+ lines.push('');++ // HNSW table+ lines.push('### 1. HNSW vs brute-force cosine');+ const h = results.hnsw;+ if (h && h.byN && !h.error) {+ lines.push(`backend: \`${JSON.stringify(h.backend)}\``);+ lines.push('');+ lines.push('| N | build ms | HNSW ms/q | brute ms/q | speedup | recall@10 |');+ lines.push('|--:|--:|--:|--:|--:|--:|');+ for (const N of ARGS.sizes) {+ const e = h.byN[N];+ if (!e) continue;+ if (e.error) { lines.push(`| ${N} | error: ${e.error} | | | | |`); continue; }+ lines.push(`| ${N} | ${e.buildMs} | ${e.hnswMsPerQuery} | ${e.bruteMsPerQuery} | ${e.speedup}x | ${e.recallAt10} |`);+ }+ } else {+ lines.push(`error: ${h?.error ?? 'no data'}`);+ }+ lines.push('');++ // Int8+ lines.push('### 2. Int8 quantization');+ const q = results.int8;+ if (q && !q.error) {+ lines.push('| metric | value |');+ lines.push('|--|--:|');+ lines.push(`| reconstruction cosine | ${q.reconstructionCosine} |`);+ lines.push(`| compression (int8 quantized) | ${q.compressionRatioInt8}x |`);+ lines.push(`| compression (base64 transport) | ${q.compressionRatioBase64}x |`);+ lines.push(`| f32 source bytes/vec | ${q.rawBytesPerVec} |`);+ lines.push(`| int8 quantized bytes/vec | ${q.quantizedRawBytesPerVec} |`);+ lines.push(`| base64 transport bytes/vec | ${q.base64BytesPerVec} |`);+ } else { lines.push(`error: ${q?.error ?? 'no data'}`); }+ lines.push('');++ // RaBitQ+ lines.push('### 3. RaBitQ');+ const rb = results.rabitq;+ if (rb && !rb.error) {+ lines.push('| metric | value |');+ lines.push('|--|--:|');+ lines.push(`| available | ${rb.available} |`);+ lines.push(`| compression ratio | ${rb.compressionRatio ?? 'null'} |`);+ lines.push(`| search ms/query | ${rb.searchMsPerQuery ?? 'null'} |`);+ if (rb.searchNote) lines.push(`| note | ${rb.searchNote} |`);+ } else { lines.push(`error: ${rb?.error ?? 'no data'}`); }+ lines.push('');++ // SONA+ lines.push('### 4. SONA WASM adapt latency (warmed)');+ const s = results.sona;+ if (s && !s.error) {+ lines.push('| metric | value |');+ lines.push('|--|--:|');+ lines.push(`| wasm available | ${s.wasmAvailable} |`);+ lines.push(`| adapt ms/call | ${s.adaptMsPerCall ?? 'null'} |`);+ if (s.iterations) lines.push(`| iterations | ${s.iterations} |`);+ if (s.targetMet_0_05ms != null) lines.push(`| < 0.05ms target met | ${s.targetMet_0_05ms} |`);+ if (s.note) lines.push(`| note | ${s.note} |`);+ } else { lines.push(`error: ${s?.error ?? 'no data'}`); }+ lines.push('');++ // MoE+ lines.push('### 5. MoE gate learning');+ const m = results.moeGate;+ if (m && !m.error) {+ lines.push('| metric | value |');+ lines.push('|--|--:|');+ lines.push(`| actions observed | ${m.actionsObserved} |`);+ lines.push(`| rewards applied | ${m.rewardsApplied} |`);+ lines.push(`| good-action Q before β after | ${m.beforeGoodQ} β ${m.afterGoodQ} (Ξ ${m.qShift}) |`);+ lines.push(`| confidence before β after | ${m.beforeConfidence} β ${m.afterConfidence} (Ξ ${m.confidenceShift}) |`);+ lines.push(`| **gate learned** | **${m.gateLearned}** |`);+ } else { lines.push(`error: ${m?.error ?? 'no data'}`); }+ lines.push('');++ // Embedding backend+ lines.push('### 6. Embedding backend (honest)');+ const eb = results.embeddingBackend;+ if (eb && !eb.error) {+ lines.push('| metric | value |');+ lines.push('|--|--|');+ lines.push(`| **backend in use** | **${eb.backend}** |`);+ lines.push(`| model string | ${eb.model} |`);+ lines.push(`| dimensions | ${eb.dimensions} |`);+ if (eb.note) lines.push(`| note | ${eb.note} |`);+ } else { lines.push(`error: ${eb?.error ?? 'no data'}`); }+ lines.push('');++ console.log(lines.join('\n'));+}++// ----------------------------------------------------------------------------+// Main+// ----------------------------------------------------------------------------+async function main() {+ log('Intelligence benchmark β measuring against built dist exports...');+ log(`dist: ${DIST}`);++ const results = {+ meta: {+ timestamp: new Date().toISOString(),+ platform: `${process.platform}/${process.arch}`,+ node: process.version,+ dist: DIST,+ dims: ARGS.dims,+ queriesPerSize: ARGS.queries,+ sizes: ARGS.sizes,+ },+ };++ // Each benchmark is isolated: a failure in one does not abort the rest.+ log('\n[1/6] HNSW vs brute-force...');+ try { results.hnsw = await benchHnsw(); } catch (e) { results.hnsw = { error: e.stack || e.message }; }++ log('\n[2/6] Int8 quantization...');+ try { results.int8 = await benchInt8(); } catch (e) { results.int8 = { error: e.stack || e.message }; }++ log('\n[3/6] RaBitQ...');+ try { results.rabitq = await benchRabitq(); } catch (e) { results.rabitq = { error: e.stack || e.message }; }++ log('\n[4/6] SONA WASM adapt...');+ try { results.sona = await benchSona(); } catch (e) { results.sona = { error: e.stack || e.message }; }++ log('\n[5/6] MoE gate learning...');+ try { results.moeGate = await benchMoeGate(); } catch (e) { results.moeGate = { error: e.stack || e.message }; }++ log('\n[6/6] Embedding backend...');+ try { results.embeddingBackend = await benchEmbeddingBackend(); } catch (e) { results.embeddingBackend = { error: e.stack || e.message }; }++ if (!ARGS.jsonOnly) printMarkdown(results);++ // Machine-readable block (always emitted, after a stable marker).+ console.log('\n===BENCH_JSON===');+ console.log(JSON.stringify(results));++ return results;+}++main()+ .then(() => process.exit(0))+ .catch((e) => { console.error('FATAL:', e.stack || e.message); process.exit(1); });diff --git a/v3/@claude-flow/cli/CLAUDE.md b/v3/@claude-flow/cli/CLAUDE.mdindex 498ece87ac..f617cc31a4 100644--- a/v3/@claude-flow/cli/CLAUDE.md+++ b/v3/@claude-flow/cli/CLAUDE.md@@ -287,7 +287,7 @@ Bash("npx @claude-flow/cli@latest hooks worker dispatch --trigger optimize")| `init` | 4 | Project initialization with wizard, presets, skills, hooks || `agent` | 8 | Agent lifecycle (spawn, list, status, stop, metrics, pool, health, logs) || `swarm` | 6 | Multi-agent swarm coordination and orchestration |-| `memory` | 11 | AgentDB memory with vector search (150x-12,500x faster) |+| `memory` | 11 | AgentDB memory with HNSW vector search (measured ~1.9xβ4.7x vs brute force above crossover) || `mcp` | 9 | MCP server management and tool execution || `task` | 6 | Task creation, assignment, and lifecycle || `session` | 7 | Session state management and persistence |@@ -308,7 +308,7 @@ Bash("npx @claude-flow/cli@latest hooks worker dispatch --trigger optimize")| `providers` | 5 | AI providers (list, add, remove, test, configure) || `plugins` | 5 | Plugin management (list, install, uninstall, enable, disable) || `deployment` | 5 | Deployment management (deploy, rollback, status, environments, release) |-| `embeddings` | 4 | Vector embeddings (embed, batch, search, init) - 75x faster with agentic-flow |+| `embeddings` | 4 | Vector embeddings (embed, batch, search, init) β agentic-flow ONNX backend (speedup unverified, no benchmark) || `claims` | 4 | Claims-based authorization (check, grant, revoke, list) || `migrate` | 5 | V2 to V3 migration with rollback support || `doctor` | 1 | System diagnostics with health checks |@@ -480,12 +480,12 @@ npx @claude-flow/cli@latest migrate validate## π§ Intelligence System (RuVector)-V3 includes the RuVector Intelligence System:-- **SONA**: Self-Optimizing Neural Architecture (<0.05ms adaptation)-- **MoE**: Mixture of Experts for specialized routing-- **HNSW**: 150x-12,500x faster pattern search+V3 includes the RuVector Intelligence System (measured numbers: see [audit](../../../docs/reviews/intelligence-system-audit-2026-05-29.md) + [`scripts/benchmark-intelligence.mjs`](../../../scripts/benchmark-intelligence.mjs)):+- **SONA**: Self-Optimizing Neural Architecture (measured 0.0043ms/adapt, target <0.05ms met)+- **MoE**: Mixture of Experts for specialized routing (gate converges β confidence 0.13β0.88 after rewards)+- **HNSW**: measured ~1.9x at N=20k, ~3.2xβ4.7x at N=5k vs brute force (recall@10 ~0.99); ANN wins above the crossover, ruvector NAPI backend (WASM not active on test host)- **EWC++**: Elastic Weight Consolidation (prevents forgetting)-- **Flash Attention**: 2.49x-7.47x speedup+- **Flash Attention**: unverified β no benchmark exists for this claimThe 4-step intelligence pipeline:1. **RETRIEVE** - Fetch relevant patterns via HNSW@@ -500,7 +500,7 @@ Features:- **Document chunking**: Configurable overlap and size- **Normalization**: L2, L1, min-max, z-score- **Hyperbolic embeddings**: PoincarΓ© ball model for hierarchical data-- **75x faster**: With agentic-flow ONNX integration+- **agentic-flow ONNX integration**: speedup unverified (no benchmark; backend reported `onnx`, model all-MiniLM-L6-v2, 384-dim)- **Neural substrate**: Integration with RuVector## π Hive-Mind Consensus@@ -520,14 +520,18 @@ Features:## V3 Performance Targets-| Metric | Target |-|--------|--------|-| Flash Attention | 2.49x-7.47x speedup |-| HNSW Search | 150x-12,500x faster |-| Memory Reduction | 50-75% with quantization |-| MCP Response | <100ms |-| CLI Startup | <500ms |-| SONA Adaptation | <0.05ms |+> Source of truth: [`docs/reviews/intelligence-system-audit-2026-05-29.md`](../../../docs/reviews/intelligence-system-audit-2026-05-29.md) + [`scripts/benchmark-intelligence.mjs`](../../../scripts/benchmark-intelligence.mjs). Numbers below are measured unless marked "target/unverified".++| Metric | Measured / Target | Status |+|--------|-------------------|--------|+| HNSW Search | ~1.9x at N=20k, ~3.2xβ4.7x at N=5k vs brute force (recall@10 ~0.99) | **Measured** (ruvector NAPI; 150x-12,500x NOT reproduced) |+| Int8 Quantization | 3.84x compression, reconstruction cosine 0.99999 | **Measured** |+| RaBitQ Quantization | 32x compression, 0.60ms/query | **Measured** |+| SONA Adaptation | 0.0043ms/adapt (target <0.05ms met) | **Measured** |+| MoE Gate | converges (confidence 0.13β0.88) | **Measured** |+| Flash Attention | 2.49x-7.47x | **Unverified** (no benchmark) |+| MCP Response | <100ms | target |+| CLI Startup | <500ms | target |## π Performance Optimization Protocoldiff --git a/v3/@claude-flow/cli/README.md b/v3/@claude-flow/cli/README.mdindex a4556f5964..d1e6974743 100644--- a/v3/@claude-flow/cli/README.md+++ b/v3/@claude-flow/cli/README.md@@ -1,15 +1,14 @@<div align="center">-[](https://flo.ruv.io/)+[](https://cognitum.one/agentic-engineering)[](https://flo.ruv.io/)[](https://goal.ruv.io/)[](https://goal.ruv.io/agents)[](https://www.npmjs.com/package/ruflo)-[](https://www.npmjs.com/package/ruflo)-[](https://www.npmjs.com/package/claude-flow)-[](https://www.npmjs.com/package/claude-flow)+[](https://github.com/ruvnet/ruflo/blob/main/data/clone-data.proof.json)+[](https://github.com/ruvnet/ruflo/blob/main/data/clone-data.ledger.json)[](https://github.com/ruvnet/claude-flow)[](https://opensource.org/licenses/MIT)@@ -73,14 +72,14 @@ There are **two different install paths** with very different surface areas. Pic# Install core + any plugins you need/plugin install ruflo-core@ruflo/plugin install ruflo-swarm@ruflo-/plugin install ruflo-autopilot@ruflo-/plugin install ruflo-federation@ruflo+/plugin install ruflo-rag-memory@ruflo+/plugin install ruflo-neural-trader@ruflo```This adds slash commands and agent definitions only. The Ruflo MCP server is NOT registered, so `memory_store`, `swarm_init`, `agent_spawn`, etc. won't be callable from Claude. For the full loop, use Path B below.<details>-<summary><strong>π All 32 plugins</strong></summary>+<summary><strong>π All 33 plugins</strong></summary>#### Core & Orchestration@@ -108,6 +107,7 @@ This adds slash commands and agent definitions only. The Ruflo MCP server is NOT| Plugin | What it does ||--------|-------------|| [**ruflo-intelligence**](plugins/ruflo-intelligence/README.md) | Agents learn from past successes and get smarter |+| [**ruflo-graph-intelligence**](plugins/ruflo-graph-intelligence/) | Sublinear graph reasoning β PageRank, delta updates, complexity-aware execution (ADR-123) || [**ruflo-daa**](plugins/ruflo-daa/README.md) | Dynamic agent behavior and cognitive patterns || [**ruflo-ruvllm**](plugins/ruflo-ruvllm/README.md) | Run local LLMs (Ollama, etc.) with smart routing || [**ruflo-goals**](plugins/ruflo-goals/README.md) | Break big goals into plans and track progress |@@ -374,13 +374,17 @@ User --> Claude Code / CLI## Documentation-Three docs for three audiences:+Four docs for four audiences:| Doc | When to read it ||-----|-----------------|| **[Status](docs/STATUS.md)** | See what currently works β capability counts, test baselines, recent fixes, what's next. The *is-it-ready* doc. || **[User Guide](docs/USERGUIDE.md)** | Daily reference β every command, every config flag, every plugin. The *how-do-I* doc. |+| **[Benchmarks](https://gist.github.com/ruvnet/298f8c668c8859b369f91734a0e9cbbe)** | v3.8.0 SOTA matrix vs LangGraph / AutoGen / CrewAI on darwin-arm64 + linux-x64. ruflo wins cold start, single turn, RSS by 1.3Γβ1953Γ. The *is-it-fast* doc. || **[Verification](verification.md)** | Cryptographically prove your installed bytes match the signed witness β `ruflo verify`. The *trust-but-verify* doc. |+| **[Team Gateway Checklist](docs/TEAM-GATEWAY-CHECKLIST.md)** | Before-merge gates, dual-mode handoff, memory namespace sharing, and witness manifest entry per merge. The *safer-team-workflows* doc. |++Benchmark internals (for reproduction): [`sota-workload-spec.md`](https://github.com/ruvnet/ruflo/blob/perf/sota-comparator-benchmarks/docs/benchmarks/sota-workload-spec.md) Β· [`SOTA-PROGRESS.md`](https://github.com/ruvnet/ruflo/blob/perf/sota-comparator-benchmarks/docs/benchmarks/SOTA-PROGRESS.md) Β· [raw matrix JSON: darwin](https://github.com/ruvnet/ruflo/blob/perf/sota-comparator-benchmarks/docs/benchmarks/sota-matrix.json) Β· [linux](https://github.com/ruvnet/ruflo/blob/perf/sota-comparator-benchmarks/docs/benchmarks/sota-matrix-linux.json)User Guide section index:diff --git a/v3/@claude-flow/cli/__tests__/bug-cluster-2219-2226.test.ts b/v3/@claude-flow/cli/__tests__/bug-cluster-2219-2226.test.tsindex d7b2fcb3c3..ad11b751b5 100644--- a/v3/@claude-flow/cli/__tests__/bug-cluster-2219-2226.test.ts+++ b/v3/@claude-flow/cli/__tests__/bug-cluster-2219-2226.test.ts@@ -24,6 +24,8 @@ import { systemTools } from '../src/mcp-tools/system-tools.js';import { hooksTools } from '../src/mcp-tools/hooks-tools.js';import { agentdbPatternStore, agentdbPatternSearch } from '../src/mcp-tools/agentdb-tools.js';import { createQLearningRouter } from '../src/ruvector/q-learning-router.js';+import { CommandParser } from '../src/parser.js';+import { routeCommand } from '../src/commands/route.js';function findTool(tools: any[], name: string) {const t = tools.find((x) => x.name === name);@@ -85,6 +87,58 @@ describe('#2222 β route feedback persists the Q-table to disk', () => {});});+describe('audit #1 β negative reward keeps its sign across all flag syntaxes', () => {+ // Repro: `route feedback -r -1.0` and `--reward -1.0` used to parse as reward=true+ // (β +1.0 after numeric coercion), so NEGATIVE feedback REINFORCED the bad agent.+ // Only the `--reward=-1.0` equals-form preserved the sign. Fix lives in the shared+ // parser (isFlagValue), so all three forms must now yield reward = -1.0.+ function parseFeedback(args: string[]) {+ const parser = new CommandParser({ allowUnknownFlags: true });+ parser.registerCommand(routeCommand);+ return parser.parse(['route', 'feedback', ...args]);+ }++ const baseArgs = ['-t', 'write tests', '-a', 'tester'];++ it('-r -1.0 (short flag, space form) β reward = -1.0', () => {+ const { flags } = parseFeedback([...baseArgs, '-r', '-1.0']);+ expect(flags.reward).toBe(-1.0);+ });++ it('--reward -1.0 (long flag, space form) β reward = -1.0', () => {+ const { flags } = parseFeedback([...baseArgs, '--reward', '-1.0']);+ expect(flags.reward).toBe(-1.0);+ });++ it('--reward=-1.0 (equals form) β reward = -1.0', () => {+ const { flags } = parseFeedback([...baseArgs, '--reward=-1.0']);+ expect(flags.reward).toBe(-1.0);+ });++ it('all three syntaxes agree (no sign-flip regression)', () => {+ const a = parseFeedback([...baseArgs, '-r', '-1.0']).flags.reward;+ const b = parseFeedback([...baseArgs, '--reward', '-1.0']).flags.reward;+ const c = parseFeedback([...baseArgs, '--reward=-1.0']).flags.reward;+ expect(a).toBe(b);+ expect(b).toBe(c);+ expect(c).toBe(-1.0);+ });++ it('positive and fractional negative values are preserved', () => {+ expect(parseFeedback([...baseArgs, '-r', '0.9']).flags.reward).toBe(0.9);+ expect(parseFeedback([...baseArgs, '-r', '-0.5']).flags.reward).toBe(-0.5);+ expect(parseFeedback([...baseArgs, '--reward', '-3.14']).flags.reward).toBe(-3.14);+ });++ it('real flags after a value-flag are still parsed as flags, not values', () => {+ // `-a tester` then `-t ...`: the -t must NOT be swallowed as the value of -a.+ const { flags } = parseFeedback(['-a', 'tester', '-t', 'write tests', '-r', '-1.0']);+ expect(flags.agent).toBe('tester');+ expect(flags.task).toBe('write tests');+ expect(flags.reward).toBe(-1.0);+ });+});+describe('#2226 β pattern store and search share a backend', () => {it('stores a pattern and finds it via search', async () => {const marker = `oauth-refresh-token-rotation-${process.pid}-${process.hrtime.bigint()}`;diff --git a/v3/@claude-flow/cli/__tests__/hooks-intelligence-learning.test.ts b/v3/@claude-flow/cli/__tests__/hooks-intelligence-learning.test.tsnew file mode 100644index 0000000000..03bbe9b551--- /dev/null+++ b/v3/@claude-flow/cli/__tests__/hooks-intelligence-learning.test.ts@@ -0,0 +1,152 @@+/**+ * Hooks Intelligence Learning β Audit Fix Smoke Tests+ *+ * AUDIT FINDINGS #4 & #5:+ * (A) hooks_intelligence_trajectory-end must feed the EWC consolidator a+ * gradient derived from the trajectory's REAL embedding (via+ * generateEmbedding) β NOT a synthetic sine wave. If no real embedding is+ * available, it must SKIP the EWC update rather than inject noise.+ * (B) hooks_intelligence_learn must actually TRIGGER a real learning/+ * consolidation cycle (distillLearning), not merely echo stats.+ *+ * These tests isolate the handlers with mocks so no real DB/ONNX is required.+ */++import { describe, it, expect, vi, beforeEach } from 'vitest';++// --- Mocks (must be declared before importing the module under test) ---++// Capture the gradient passed to EWC so we can assert it is the real embedding.+const recordGradient = vi.fn();+const getConsolidationStats = vi.fn(() => ({+ avgPenalty: 0.42,+ consolidationCount: 1,+ highImportancePatterns: 0,+ totalPatterns: 1,+}));++vi.mock('../src/memory/ewc-consolidation.js', () => ({+ getEWCConsolidator: vi.fn(async () => ({+ recordGradient,+ getConsolidationStats,+ })),+}));++// SONA optimizer present and "learns" so the success branch runs.+vi.mock('../src/memory/sona-optimizer.js', () => ({+ getSONAOptimizer: vi.fn(async () => ({+ processTrajectoryOutcome: vi.fn(() => ({+ learned: true,+ patternKey: 'pattern-x',+ confidence: 0.9,+ })),+ getStats: vi.fn(() => ({+ totalPatterns: 3,+ successfulRoutings: 2,+ failedRoutings: 1,+ trajectoriesProcessed: 3,+ avgConfidence: 0.7,+ })),+ })),+}));++// REAL embedding signal β a distinctive vector that is clearly NOT a sine wave.+const REAL_EMBEDDING = new Array(384).fill(0).map((_, i) => (i % 2 === 0 ? 0.5 : 0.25));+vi.mock('../src/memory/memory-initializer.js', () => ({+ generateEmbedding: vi.fn(async () => ({+ embedding: REAL_EMBEDDING,+ dimensions: REAL_EMBEDDING.length,+ model: 'mock-onnx',+ })),+ // storeEntry must succeed so the success/learning branch (which contains the+ // EWC update) is reached in trajectory-end.+ storeEntry: vi.fn(async () => ({ success: true, id: 'mock-trajectory-id' })),+}));++// Intelligence layer β distillLearning is the real DISTILL+CONSOLIDATE path.+const distillLearning = vi.fn(async () => ({ patternsDistilled: 5, ewcPenalty: 0.13 }));+const runBackgroundLearning = vi.fn(async () => {});+vi.mock('../src/memory/intelligence.js', () => ({+ distillLearning,+ runBackgroundLearning,+}));++// graph-edge-writer is fire-and-forget; stub it.+vi.mock('../src/memory/graph-edge-writer.js', () => ({+ insertGraphEdge: vi.fn(async () => {}),+}));++import {+ hooksTrajectoryStart,+ hooksTrajectoryStep,+ hooksTrajectoryEnd,+ hooksIntelligenceLearn,+} from '../src/mcp-tools/hooks-tools.js';+import { generateEmbedding } from '../src/memory/memory-initializer.js';++describe('AUDIT FIX #4 β trajectory-end feeds REAL embedding gradient to EWC', () => {+ beforeEach(() => {+ recordGradient.mockClear();+ (generateEmbedding as ReturnType<typeof vi.fn>).mockClear();+ });++ it('records a gradient equal to the real trajectory embedding, not a sine wave', async () => {+ const started = (await hooksTrajectoryStart.handler({ task: 'do work', agent: 'coder' })) as any;+ const trajectoryId = started.trajectoryId as string;++ await hooksTrajectoryStep.handler({ trajectoryId, action: 'edit-file', result: 'ok', quality: 0.9 });++ const ended = (await hooksTrajectoryEnd.handler({ trajectoryId, success: true })) as any;++ // generateEmbedding was used to derive the gradient (mirrors DISTILL path)+ expect(generateEmbedding).toHaveBeenCalledTimes(1);++ // EWC received the REAL embedding, not synthetic noise+ expect(recordGradient).toHaveBeenCalledTimes(1);+ const passedGradient = recordGradient.mock.calls[0][1] as number[];+ expect(passedGradient).toEqual(REAL_EMBEDDING);++ // Guard: the OLD synthetic gradient was sin(i*0.01)*(steps/10). With 1 step+ // that produced values like sin(0)=0, sin(0.01)*0.1 ~= 0.001 β confirm the+ // gradient is NOT that degenerate sine sequence.+ expect(passedGradient[0]).not.toBeCloseTo(Math.sin(0) * (1 / 10), 6);+ expect(passedGradient[1]).not.toBeCloseTo(Math.sin(0.01) * (1 / 10), 6);++ expect(ended.learning.ewcConsolidation).toBe(true);+ });++ it('SKIPS the EWC update (no synthetic gradient) when no real embedding is available', async () => {+ (generateEmbedding as ReturnType<typeof vi.fn>).mockResolvedValueOnce({+ embedding: [],+ dimensions: 0,+ model: 'none',+ });++ const started = (await hooksTrajectoryStart.handler({ task: 'task2', agent: 'coder' })) as any;+ const trajectoryId = started.trajectoryId as string;+ await hooksTrajectoryStep.handler({ trajectoryId, action: 'a', result: 'ok' });++ const ended = (await hooksTrajectoryEnd.handler({ trajectoryId, success: true })) as any;++ // No gradient recorded β we skip rather than inject noise.+ expect(recordGradient).not.toHaveBeenCalled();+ expect(ended.learning.ewcConsolidation).toBe(false);+ });+});++describe('AUDIT FIX #5 β hooks_intelligence_learn actually triggers a learning cycle', () => {+ beforeEach(() => {+ distillLearning.mockClear();+ runBackgroundLearning.mockClear();+ });++ it('calls the real distillLearning (DISTILL+CONSOLIDATE) path, not just stats echo', async () => {+ const result = (await hooksIntelligenceLearn.handler({ consolidate: true })) as any;++ expect(distillLearning).toHaveBeenCalledTimes(1);+ expect(result.cycleTriggered).toBe(true);+ expect(result.patternsDistilled).toBe(5);+ expect(result.implementation).toBe('real-distill-consolidate');+ expect(result.learned).toBe(true);+ });+});diff --git a/v3/@claude-flow/cli/__tests__/memory-ruvector-deep.test.ts b/v3/@claude-flow/cli/__tests__/memory-ruvector-deep.test.tsindex 35745993dc..de3ef5be2b 100644--- a/v3/@claude-flow/cli/__tests__/memory-ruvector-deep.test.ts+++ b/v3/@claude-flow/cli/__tests__/memory-ruvector-deep.test.ts@@ -257,6 +257,61 @@ describe('Memory Initializer', () => {expect(status.dimensions).toBe(384);});});++ // AUDIT #3: generateEmbedding must expose a truthful `backend` field so an+ // operator can distinguish real ONNX semantics from the deterministic hash+ // fallback (inverted/meaningless semantics) even when `model` reports a+ // real-looking name.+ describe('generateEmbedding backend field', () => {+ const prevDisableBridge = process.env.CLAUDE_FLOW_DISABLE_BRIDGE;++ afterEach(() => {+ vi.resetModules();+ vi.doUnmock('@huggingface/transformers');+ vi.doUnmock('@xenova/transformers');+ vi.doUnmock('ruvector');+ vi.doUnmock('agentic-flow');+ vi.doUnmock('agentic-flow/reasoningbank');+ if (prevDisableBridge === undefined) delete process.env.CLAUDE_FLOW_DISABLE_BRIDGE;+ else process.env.CLAUDE_FLOW_DISABLE_BRIDGE = prevDisableBridge;+ });++ it("reports backend='mock' when no real embedding model is available", async () => {+ // Force the raw fallback path (no AgentDB bridge) and make every real+ // embedding provider import fail so loadEmbeddingModel lands on the+ // hash fallback (model = null).+ process.env.CLAUDE_FLOW_DISABLE_BRIDGE = '1';+ vi.resetModules();+ const fail = () => { throw new Error('unavailable in test'); };+ vi.doMock('@huggingface/transformers', fail);+ vi.doMock('@xenova/transformers', fail);+ vi.doMock('ruvector', fail);+ vi.doMock('agentic-flow', fail);+ vi.doMock('agentic-flow/reasoningbank', fail);++ const { generateEmbedding } = await import('../src/memory/memory-initializer.js');+ const result = await generateEmbedding('audit-3 mock-backend assertion');++ expect(result.backend).toBe('mock');+ expect(result.model).toBe('hash-fallback');+ expect(Array.isArray(result.embedding)).toBe(true);+ expect(result.embedding.length).toBe(result.dimensions);+ });++ it("always sets backend, and 'mock' implies the hash-fallback model", async () => {+ // Invariant check that holds regardless of which providers are installed:+ // backend is one of the two known values, and mock <=> hash-fallback.+ process.env.CLAUDE_FLOW_DISABLE_BRIDGE = '1';+ vi.resetModules();+ const { generateEmbedding } = await import('../src/memory/memory-initializer.js');+ const result = await generateEmbedding('audit-3 backend invariant');++ expect(['onnx', 'mock']).toContain(result.backend);+ if (result.backend === 'mock') {+ expect(result.model).toBe('hash-fallback');+ }+ }, 60000); // real ONNX model can be slow to cold-load when installed+ });});// =============================================================================diff --git a/v3/@claude-flow/cli/package.json b/v3/@claude-flow/cli/package.jsonindex 119741d0d0..8b41c8d7eb 100644--- a/v3/@claude-flow/cli/package.json+++ b/v3/@claude-flow/cli/package.json@@ -1,6 +1,6 @@{"name": "@claude-flow/cli",- "version": "3.10.6",+ "version": "3.10.7","type": "module","description": "Ruflo CLI - Enterprise AI agent orchestration with 60+ specialized agents, swarm coordination, MCP server, self-learning hooks, and vector memory for Claude Code","main": "dist/src/index.js",diff --git a/v3/@claude-flow/cli/src/mcp-tools/hooks-tools.ts b/v3/@claude-flow/cli/src/mcp-tools/hooks-tools.tsindex 4f1c7ff9a4..feea4716be 100644--- a/v3/@claude-flow/cli/src/mcp-tools/hooks-tools.ts+++ b/v3/@claude-flow/cli/src/mcp-tools/hooks-tools.ts@@ -2597,17 +2597,35 @@ export const hooksTrajectoryEnd: MCPTool = {const ewc = await getEWCConsolidator();if (ewc) {try {- // Record gradient sample for Fisher matrix update- // Create a simple gradient from trajectory steps- const gradients = new Array(384).fill(0).map((_, i) =>- Math.sin(i * 0.01) * (trajectory.steps.length / 10)- );- ewc.recordGradient(`trajectory-${trajectoryId}`, gradients, success);- const stats = ewc.getConsolidationStats();- ewcResult = {- consolidated: true,- penalty: stats.avgPenalty,- };+ // AUDIT FIX #4: derive a REAL gradient from the trajectory's+ // embedding (mirrors the DISTILL path, where step content is+ // embedded via generateEmbedding) instead of a synthetic sine+ // wave. The EWC library treats the embedding as the gradient+ // proxy (see recordPatternOutcome in ewc-consolidation.ts).+ let gradients: number[] | null = null;+ try {+ const { generateEmbedding } = await import('../memory/memory-initializer.js');+ // Embed the same summary that was persisted for semantic search,+ // so the Fisher update reflects the actual recorded trajectory.+ const summary = `Task: ${trajectory.task} | Agent: ${trajectory.agent} | Steps: ${trajectory.steps.map(s => `${s.action}=>${s.result}`).join('; ')}${feedback ? ` | Feedback: ${feedback}` : ''}`;+ const embeddingResult = await generateEmbedding(summary);+ if (embeddingResult?.embedding && embeddingResult.embedding.length > 0) {+ gradients = embeddingResult.embedding;+ }+ } catch {+ // Embedding generation unavailable β fall through and skip EWC+ }++ if (gradients) {+ ewc.recordGradient(`trajectory-${trajectoryId}`, gradients, success);+ const stats = ewc.getConsolidationStats();+ ewcResult = {+ consolidated: true,+ penalty: stats.avgPenalty,+ };+ }+ // If no real embedding-derived gradient is available, SKIP the EWC+ // update rather than feeding the Fisher matrix synthetic noise.} catch {// EWC consolidation failed, continue without it}@@ -3057,7 +3075,26 @@ export const hooksIntelligenceLearn: MCPTool = {const consolidate = params.consolidate !== false;const startTime = Date.now();- // Get SONA statistics+ // AUDIT FIX #5: actually TRIGGER a learning/consolidation cycle instead of+ // only reading and echoing stats. This calls the real DISTILL path+ // (LoRA-style confidence updates with EWC++ consolidation protection) and+ // the background learning pass, then reports the resulting stats.+ let distill: { patternsDistilled: number; ewcPenalty: number } | null = null;+ let distillTriggered = false;+ try {+ const intelligence = await import('../memory/intelligence.js');+ // DISTILL + CONSOLIDATE: real LoRA update with EWC++ protection+ distill = await intelligence.distillLearning();+ distillTriggered = distill !== null;+ // Run background learning (ruvllm) pass as well β best-effort+ try {+ await intelligence.runBackgroundLearning();+ } catch { /* best-effort */ }+ } catch {+ // intelligence layer unavailable β fall back to stats-only reporting+ }++ // Get SONA statistics (AFTER triggering the cycle so they reflect the update)let sonaStats = {totalPatterns: 0,successfulRoutings: 0,@@ -3077,7 +3114,7 @@ export const hooksIntelligenceLearn: MCPTool = {};}- // Get EWC++ statistics and optionally trigger consolidation+ // Get EWC++ statistics after the consolidation cycle ranlet ewcStats = {consolidation: false,fisherUpdated: false,@@ -3092,17 +3129,21 @@ export const hooksIntelligenceLearn: MCPTool = {consolidation: true,fisherUpdated: stats.consolidationCount > 0,forgettingPrevented: stats.highImportancePatterns,- avgPenalty: stats.avgPenalty,+ avgPenalty: distill?.ewcPenalty ?? stats.avgPenalty,};}}return {- learned: sonaStats.totalPatterns > 0,+ // "learned" now reflects whether a real distill cycle actually ran+ learned: distillTriggered || sonaStats.totalPatterns > 0,+ cycleTriggered: distillTriggered,+ patternsDistilled: distill?.patternsDistilled ?? 0,duration: Date.now() - startTime,updates: {trajectoriesProcessed: sonaStats.trajectoriesProcessed,patternsLearned: sonaStats.totalPatterns,+ patternsDistilled: distill?.patternsDistilled ?? 0,successRate: sonaStats.trajectoriesProcessed > 0? (sonaStats.successfulRoutings / (sonaStats.successfulRoutings + sonaStats.failedRoutings) * 100).toFixed(1) + '%': '0%',@@ -3112,7 +3153,9 @@ export const hooksIntelligenceLearn: MCPTool = {average: sonaStats.avgConfidence,implementation: sona ? 'real-sona' : 'not-available',},- implementation: sona ? 'real-sona-learning' : 'placeholder',+ implementation: distillTriggered+ ? 'real-distill-consolidate'+ : (sona ? 'real-sona-learning' : 'placeholder'),};},};diff --git a/v3/@claude-flow/cli/src/mcp-tools/memory-tools.ts b/v3/@claude-flow/cli/src/mcp-tools/memory-tools.tsindex 9d22a2eced..b598a1b92a 100644--- a/v3/@claude-flow/cli/src/mcp-tools/memory-tools.ts+++ b/v3/@claude-flow/cli/src/mcp-tools/memory-tools.ts@@ -926,6 +926,16 @@ export const memoryTools: MCPTool[] = [}}+ // AUDIT #3: report the embedding backend truthfully β a hash-fallback+ // import is NOT semantically searchable, so an operator must not read+ // "ONNX ... (384-dim)" when the vectors are mock.+ let importBackend: 'onnx' | 'mock' | 'unknown' = 'unknown';+ try {+ const { generateEmbedding } = await import('../memory/memory-initializer.js');+ const probe = await generateEmbedding('memory_import_claude backend probe');+ importBackend = probe.backend ?? 'unknown';+ } catch { /* probe failed β leave 'unknown' */ }+return {success: true,imported,@@ -935,7 +945,8 @@ export const memoryTools: MCPTool[] = [files: memoryFiles.length,projects: projects.size,namespace: ns,- embedding: 'ONNX all-MiniLM-L6-v2 (384-dim)',+ embedding: `all-MiniLM-L6-v2 (384-dim, backend=${importBackend})`,+ embeddingBackend: importBackend,};},},@@ -1001,14 +1012,37 @@ export const memoryTools: MCPTool[] = [if (stats) intelligence = { sonaEnabled: stats.sonaEnabled, patternsLearned: stats.patternsLearned, trajectoriesRecorded: stats.trajectoriesRecorded };} catch { /* not initialized */ }+ // AUDIT #3: probe the embedding backend so operators can tell real ONNX+ // output from the deterministic hash fallback (which has inverted/+ // meaningless semantics). Without this, the status string reports the+ // model name unconditionally and mock output is indistinguishable.+ let embeddingBackend: 'onnx' | 'mock' | 'unknown' = 'unknown';+ try {+ const { generateEmbedding } = await import('../memory/memory-initializer.js');+ const probe = await generateEmbedding('memory_bridge_status backend probe');+ embeddingBackend = probe.backend ?? 'unknown';+ } catch { /* probe failed β leave 'unknown' */ }++ const embeddingLabel = `all-MiniLM-L6-v2 (384-dim, backend=${embeddingBackend})`;+return {claudeCode: { memoryFiles: claudeFiles, projects: claudeProjects },- agentdb: { totalEntries: agentdbEntries, claudeMemoryEntries, namespaces: namespaceCounts, backend: 'sql.js + ONNX' },+ agentdb: {+ totalEntries: agentdbEntries,+ claudeMemoryEntries,+ namespaces: namespaceCounts,+ backend: embeddingBackend === 'mock' ? 'sql.js + MOCK (hash fallback)' : 'sql.js + ONNX',+ embeddingBackend,+ },intelligence,// #1940: report 'connected' whenever ANY namespace has imported// content, not just `claude-memories` β the bridge can be in active// use from other import paths (e.g. plugin namespaces, task memory).- bridge: { status: agentdbEntries > 0 ? 'connected' : 'not-synced', embedding: 'all-MiniLM-L6-v2 (384-dim)' },+ bridge: {+ status: agentdbEntries > 0 ? 'connected' : 'not-synced',+ embedding: embeddingLabel,+ embeddingBackend,+ },};},},diff --git a/v3/@claude-flow/cli/src/memory/memory-bridge.ts b/v3/@claude-flow/cli/src/memory/memory-bridge.tsindex f339958374..5361339b96 100644--- a/v3/@claude-flow/cli/src/memory/memory-bridge.ts+++ b/v3/@claude-flow/cli/src/memory/memory-bridge.ts@@ -1078,7 +1078,7 @@ export async function bridgeDeleteEntry(options: {export async function bridgeGenerateEmbedding(text: string,dbPath?: string,-): Promise<{ embedding: number[]; dimensions: number; model: string } | null> {+): Promise<{ embedding: number[]; dimensions: number; model: string; backend?: 'onnx' | 'mock' } | null> {const registry = await getRegistry(dbPath);if (!registry) return null;@@ -1090,10 +1090,16 @@ export async function bridgeGenerateEmbedding(const emb = await embedder.embed(text);if (!emb) return null;+ // AUDIT #3: surface backend truthfully. AgentDB's embedder is a real ONNX+ // model when present; if it ever exposes a mock/stub signal, honor it.+ const isMock = (embedder as { isMock?: boolean; backend?: string }).isMock === true+ || (embedder as { backend?: string }).backend === 'mock';+return {embedding: Array.from(emb),dimensions: emb.length,model: 'Xenova/all-MiniLM-L6-v2',+ backend: isMock ? 'mock' : 'onnx',};} catch {return null;diff --git a/v3/@claude-flow/cli/src/memory/memory-initializer.ts b/v3/@claude-flow/cli/src/memory/memory-initializer.tsindex 24bf4b694f..8ee4d8da46 100644--- a/v3/@claude-flow/cli/src/memory/memory-initializer.ts+++ b/v3/@claude-flow/cli/src/memory/memory-initializer.ts@@ -1837,17 +1837,34 @@ export async function loadEmbeddingModel(options?: {/*** Generate real embedding for text* Uses ONNX model if available, falls back to deterministic hash+ *+ * AUDIT #3: the `backend` field is the authoritative signal for whether the+ * returned vector carries real ONNX semantics ('onnx') or the deterministic+ * hash fallback ('mock'). The hash fallback produces inverted/meaningless+ * semantics, so operators MUST be able to tell the two apart even when the+ * `model` string reports a real model name (e.g. the AgentDB bridge always+ * labels its output 'Xenova/all-MiniLM-L6-v2' regardless of whether AgentDB's+ * own embedder is real or stubbed). Set `backend` truthfully by the path that+ * actually produced the vector. Do NOT change the embedding math.*/export async function generateEmbedding(text: string): Promise<{embedding: number[];dimensions: number;model: string;+ backend: 'onnx' | 'mock';}> {// ADR-053: Try AgentDB v3 bridge firstconst bridge = await getBridge();if (bridge) {const bridgeResult = await bridge.bridgeGenerateEmbedding(text);- if (bridgeResult) return bridgeResult;+ if (bridgeResult) {+ // The bridge labels its output with a real model name unconditionally;+ // honor the backend it reports if present, otherwise treat a real model+ // name as ONNX (the bridge only returns when AgentDB's embedder exists).+ const backend: 'onnx' | 'mock' =+ (bridgeResult as { backend?: 'onnx' | 'mock' }).backend ?? 'onnx';+ return { ...bridgeResult, backend };+ }}// Ensure model is loaded@@ -1869,7 +1886,8 @@ export async function generateEmbedding(text: string): Promise<{return {embedding,dimensions: embedding.length,- model: 'onnx'+ model: 'onnx',+ backend: 'onnx'};}} catch {@@ -1877,12 +1895,14 @@ export async function generateEmbedding(text: string): Promise<{}}- // Deterministic hash-based fallback (for testing/demo without ONNX)+ // Deterministic hash-based fallback (for testing/demo without ONNX).+ // AUDIT #3: backend='mock' β these vectors do NOT carry real semantics.const embedding = generateHashEmbedding(text, state.dimensions);return {embedding,dimensions: state.dimensions,- model: 'hash-fallback'+ model: 'hash-fallback',+ backend: 'mock'};}diff --git a/v3/@claude-flow/cli/src/parser.ts b/v3/@claude-flow/cli/src/parser.tsindex 6eea07b2a1..f525b2be49 100644--- a/v3/@claude-flow/cli/src/parser.ts+++ b/v3/@claude-flow/cli/src/parser.ts@@ -298,7 +298,7 @@ export class CommandParser {if (booleanFlags.has(normalizedKey)) {flags[normalizedKey] = true;- } else if (nextIndex < args.length && !args[nextIndex].startsWith('-')) {+ } else if (nextIndex < args.length && this.isFlagValue(args[nextIndex])) {flags[normalizedKey] = this.parseValue(args[nextIndex]);nextIndex++;} else {@@ -316,7 +316,7 @@ export class CommandParser {if (booleanFlags.has(normalizedKey)) {flags[normalizedKey] = true;- } else if (nextIndex < args.length && !args[nextIndex].startsWith('-')) {+ } else if (nextIndex < args.length && this.isFlagValue(args[nextIndex])) {flags[normalizedKey] = this.parseValue(args[nextIndex]);nextIndex++;} else {@@ -334,6 +334,27 @@ export class CommandParser {return { flags, nextIndex };}+ /**+ * Decide whether `arg` should be consumed as the VALUE of the preceding flag,+ * rather than treated as the next flag.+ *+ * Bug fix (audit #1, follow-up to #2222): a negative numeric value such as+ * `-1.0` starts with '-', so the old `!arg.startsWith('-')` test rejected it+ * as a value and parsed it as a (bogus) short flag. For `route feedback+ * -r -1.0` this silently dropped the value and coerced reward to `true` β 1.0,+ * so NEGATIVE feedback REINFORCED the agent. Only `--reward=-1.0` worked.+ *+ * Anything not starting with '-' is a value (unchanged). Anything that starts+ * with '-' is a value ONLY if it is a pure negative number (e.g. `-1`, `-1.0`,+ * `-3.14`, `-1e3`). Real flags like `-r`, `--reward`, `-abc` are never numeric+ * after the leading dash, so they are still correctly treated as flags.+ */+ private isFlagValue(arg: string): boolean {+ if (!arg.startsWith('-')) return true;+ // Negative number: '-' followed by a parseable numeric literal.+ return /^-\d*\.?\d+(?:[eE][+-]?\d+)?$/.test(arg);+ }+private parseValue(value: string): string | number | boolean {// Booleanif (value.toLowerCase() === 'true') return true;diff --git a/v3/@claude-flow/cli/src/ruvector/vector-db.ts b/v3/@claude-flow/cli/src/ruvector/vector-db.tsindex 72960b7d13..9b857808bb 100644--- a/v3/@claude-flow/cli/src/ruvector/vector-db.ts+++ b/v3/@claude-flow/cli/src/ruvector/vector-db.ts@@ -11,6 +11,10 @@* Created with love by ruv.io*/+import os from 'node:os';+import path from 'node:path';+import { randomUUID } from 'node:crypto';+// ============================================================================// Types// ============================================================================@@ -172,7 +176,29 @@ export async function loadRuVector(): Promise<boolean> {const VectorDBClass = ruvector.VectorDB || ruvector.VectorDb;ruvectorModule = {createVectorDB: async (dimensions: number): Promise<VectorDB> => {- const db = new VectorDBClass({ dimensions });+ // Build the HNSW graph with explicit, high-recall parameters instead+ // of relying on the native layer's undocumented defaults. Measured on+ // a 384-dim clustered corpus (scripts/benchmark-intelligence.mjs):+ // - sparser graphs (e.g. m=16/efConstruction=64) are faster but+ // drop recall@10 to ~0.68 at N=20k β unacceptable for a memory+ // store, so we use the standard high-recall HNSW setting+ // m=32/efConstruction=200, which keeps recall@10 ~0.99 while still+ // scaling sub-linearly (log-graph traversal) above the crossover.+ // These map onto ruvector's DbOptions.hnsw { m, efConstruction }.+ //+ // storagePath: the native ruvector DB takes an exclusive file lock on+ // its storage path. Without an explicit path it defaults to a shared+ // file in the CWD (e.g. agentdb.rvf), which is frequently already+ // held by a running daemon/MCP server. When the lock cannot be+ // acquired the constructor THROWS, and createVectorDB() below+ // silently degrades to the O(n) brute-force FallbackVectorDB β so the+ // "HNSW" index is never actually used. Giving each transient index a+ // unique path avoids that contention so the real HNSW graph is built.+ const db = new VectorDBClass({+ dimensions,+ hnswConfig: { m: 32, efConstruction: 200 },+ storagePath: path.join(os.tmpdir(), `ruvector-${process.pid}-${randomUUID()}.rvf`),+ });// Wrap ruvector's VectorDB to match our interfacereturn {insert: (embedding: Float32Array, id: string, metadata?: Record<string, unknown>) => {@@ -248,8 +274,13 @@ export async function createVectorDB(dimensions: number = 768): Promise<VectorDBif (ruvectorModule && typeof ruvectorModule.createVectorDB === 'function') {try {return await ruvectorModule.createVectorDB(dimensions);- } catch {- // Fall back to simple implementation+ } catch (err) {+ // Fall back to simple implementation, but make the degradation VISIBLE.+ // A silent fall-through here turns HNSW into O(n) brute force without any+ // signal β exactly the failure mode that masked the lock-contention bug.+ console.warn(+ `[vector-db] ruvector HNSW init failed, falling back to brute-force search: ${(err as Error)?.message ?? err}`+ );}}diff --git a/v3/@claude-flow/integration/src/attention-coordinator.ts b/v3/@claude-flow/integration/src/attention-coordinator.tsindex a62d81dd8b..86dded4c06 100644--- a/v3/@claude-flow/integration/src/attention-coordinator.ts+++ b/v3/@claude-flow/integration/src/attention-coordinator.ts@@ -2,8 +2,9 @@* Attention Coordinator for Flash Attention Integration** Provides integration with agentic-flow's attention mechanisms,- * including Flash Attention for 2.49x-7.47x speedup with- * 50-75% memory reduction.+ * including an approximate sparse Flash Attention path. Speedup and+ * memory reduction are unverified β no benchmark kernel is wired here.+ * See docs/reviews/intelligence-system-audit-2026-05-29.md** Supported Mechanisms:* - Flash Attention (fastest, recommended)@@ -69,8 +70,11 @@ const MECHANISM_PROFILES: Record<AttentionMechanism, {bestFor: string[];}> = {'flash': {- speedupRange: [2.49, 7.47],- memoryReduction: 0.75,+ // Speedup is UNMEASURED β sentinel [0, 0] means "no verified benchmark".+ // Do NOT restore a fabricated 2.49x-7.47x range.+ // See docs/reviews/intelligence-system-audit-2026-05-29.md+ speedupRange: [0, 0],+ memoryReduction: 0,latencyMs: [0.7, 1.5],bestFor: ['general', 'high-throughput', 'memory-constrained'],},@@ -158,7 +162,8 @@ export class AttentionCoordinator extends EventEmitter {* This implements ADR-001: Adopt agentic-flow as Core Foundation* When a reference is provided, attention computation for sequences* longer than 512 tokens delegates to agentic-flow's optimized- * Flash Attention implementation for 2.49x-7.47x speedup.+ * Flash Attention implementation (approximate sparse attention;+ * speedup unverified β see docs/reviews/intelligence-system-audit-2026-05-29.md).** @param attentionRef - The agentic-flow Attention interface reference*/@@ -496,8 +501,8 @@ export class AttentionCoordinator extends EventEmitter {* Perform attention computation** ADR-001: For sequences longer than 512 tokens, delegates to- * agentic-flow's native Flash Attention for 2.49x-7.47x speedup- * and 50-75% memory reduction.+ * agentic-flow's native Flash Attention (approximate sparse attention;+ * speedup unverified β see docs/reviews/intelligence-system-audit-2026-05-29.md).*/private async performAttention(params: {query: number[] | Float32Array;@@ -514,8 +519,9 @@ export class AttentionCoordinator extends EventEmitter {// Calculate sequence length for delegation decisionconst sequenceLength = qArray.length;- // ADR-001: Delegate to agentic-flow for long sequences- // Flash Attention provides 2.49x-7.47x speedup for sequences > 512 tokens+ // ADR-001: Delegate to agentic-flow for long sequences.+ // Flash Attention is an approximate sparse path; speedup unverified β+ // see docs/reviews/intelligence-system-audit-2026-05-29.mdif (this.isDelegationEnabled() &&this.agenticFlowAttention &&@@ -631,9 +637,15 @@ export class AttentionCoordinator extends EventEmitter {this.metrics.throughputTps = 1000 / this.metrics.avgLatencyMs;}- // Calculate speedup factor based on mechanism+ // Speedup factor based on mechanism profile.+ // A [0, 0] speedupRange is the "unmeasured" sentinel (e.g. flash, where+ // no benchmark kernel is wired) β report 0 rather than fabricate a value.+ // See docs/reviews/intelligence-system-audit-2026-05-29.mdconst profile = MECHANISM_PROFILES[this.config.mechanism];- this.metrics.speedupFactor = (profile.speedupRange[0] + profile.speedupRange[1]) / 2;+ const isUnmeasured = profile.speedupRange[0] === 0 && profile.speedupRange[1] === 0;+ this.metrics.speedupFactor = isUnmeasured+ ? 0 // unmeasured β no fabrication+ : (profile.speedupRange[0] + profile.speedupRange[1]) / 2;this.metrics.memoryEfficiency = 1 - profile.memoryReduction;}diff --git a/v3/@claude-flow/swarm/src/attention-coordinator.ts b/v3/@claude-flow/swarm/src/attention-coordinator.tsindex 8e1aac45b4..0c6d840dd5 100644--- a/v3/@claude-flow/swarm/src/attention-coordinator.ts+++ b/v3/@claude-flow/swarm/src/attention-coordinator.ts@@ -3,15 +3,15 @@** Implements attention-based coordination mechanisms from agentic-flow@alpha:* - multi-head: Standard multi-head attention- * - flash: 2.49x-7.47x speedup, 75% memory reduction+ * - flash: approximate sparse attention; speedup unverified β see docs/reviews/intelligence-system-audit-2026-05-29.md* - linear: For long sequences* - hyperbolic: Hierarchical data* - moe: Mixture of Experts routing* - graph-rope: Graph-aware positional embeddings** Performance Targets:- * - Flash Attention: 2.49x-7.47x speedup- * - Memory Reduction: 50-75%+ * - Flash Attention: approximate sparse attention; speedup unverified β see docs/reviews/intelligence-system-audit-2026-05-29.md+ * - Memory Reduction: unverified β see docs/reviews/intelligence-system-audit-2026-05-29.md* - MoE Routing: <5ms** @module v3/swarm/attention-coordinator@@ -28,7 +28,7 @@ import { EventEmitter } from 'events';*/export type AttentionType =| 'multi-head' // Standard multi-head attention- | 'flash' // 2.49x-7.47x speedup, 75% memory reduction+ | 'flash' // approximate sparse attention; speedup unverified β see docs/reviews/intelligence-system-audit-2026-05-29.md| 'linear' // For long sequences| 'hyperbolic' // Hierarchical data| 'moe' // Mixture of Experts@@ -374,7 +374,8 @@ export class AttentionCoordinator extends EventEmitter {// ===========================================================================/**- * Flash Attention - 2.49x-7.47x speedup+ * Flash Attention - approximate sparse attention; speedup unverified+ * β see docs/reviews/intelligence-system-audit-2026-05-29.md*/private async flashAttentionCoordination(agentOutputs: AgentOutput[]@@ -418,8 +419,11 @@ export class AttentionCoordinator extends EventEmitter {memoryUsed,participatingAgents: agentOutputs.map(o => o.agentId),metadata: {- speedup: '2.49x-7.47x',- memoryReduction: '75%',+ // Speedup/memory reduction are UNMEASURED β no benchmark is wired.+ // Do not advertise a made-up factor. See+ // docs/reviews/intelligence-system-audit-2026-05-29.md+ speedup: 'unverified',+ memoryReduction: 'unverified',blockSize,},};@@ -967,10 +971,13 @@ export class AttentionCoordinator extends EventEmitter {this.performanceStats.totalLatency += latency;if (mechanism === 'flash') {- // Track Flash Attention performance- // In production, compare against baseline- this.performanceStats.flashSpeedup = 2.49 + Math.random() * 4.98; // 2.49x-7.47x- this.performanceStats.memoryReduction = 0.75;+ // Flash Attention speedup is UNMEASURED in this build β no benchmark+ // kernel is wired here. Do NOT fabricate a value. Sentinel 0 means+ // "unmeasured"; consumers must not advertise a speedup until a real+ // measured path (kernel benchmark vs baseline) is implemented.+ // See docs/reviews/intelligence-system-audit-2026-05-29.md+ this.performanceStats.flashSpeedup = 0; // 0 = unmeasured (no fabrication)+ this.performanceStats.memoryReduction = 0; // 0 = unmeasured (no fabrication)}}diff --git a/v3/CLAUDE.md b/v3/CLAUDE.mdindex 6a5c02c45b..067c0c946c 100644--- a/v3/CLAUDE.md+++ b/v3/CLAUDE.md@@ -31,10 +31,14 @@ npm install && npm run build && npm test## Performance Targets-| Metric | Target | Status |-|--------|--------|--------|-| HNSW Search | 150x-12,500x faster | Implemented |-| Memory Reduction | 50-75% (Int8 quantization) | Implemented |-| MCP Response | <100ms | Achieved |-| CLI Startup | <500ms | Achieved |-| Flash Attention | 2.49x-7.47x speedup | In progress |+> Source of truth: [`docs/reviews/intelligence-system-audit-2026-05-29.md`](../docs/reviews/intelligence-system-audit-2026-05-29.md) + [`scripts/benchmark-intelligence.mjs`](../scripts/benchmark-intelligence.mjs). Numbers below are measured unless marked "target/unverified".++| Metric | Measured / Target | Status |+|--------|-------------------|--------|+| HNSW Search | ~1.9x at N=20k, ~3.2xβ4.7x at N=5k vs brute force (recall@10 ~0.99) | **Measured** (ruvector NAPI; 150x-12,500x NOT reproduced) |+| Int8 Quantization | 3.84x compression, reconstruction cosine 0.99999 | **Measured** |+| RaBitQ Quantization | 32x compression, 0.60ms/query | **Measured** |+| SONA Adaptation | 0.0043ms/adapt (target <0.05ms met) | **Measured** |+| MCP Response | <100ms | target |+| CLI Startup | <500ms | target |+| Flash Attention | 2.49x-7.47x | **Unverified** (no benchmark) |