Skip to content

armgit / ruflo

Public Imported from GitHub
0

🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated

dream(performance): #3129 remove unwired diskann-backend.ts, fix stale HNSW claims in generated CLAUDE.md (evaluated, ACCEPT) (#3130)

rUv0db381f7f781
diff --git a/docs/dream-cycle/LEDGER.md b/docs/dream-cycle/LEDGER.md
index 172993c700..3ef61cbe0b 100644
--- a/docs/dream-cycle/LEDGER.md
+++ b/docs/dream-cycle/LEDGER.md
@@ -169,7 +169,7 @@ as a first-class, high-priority candidate, not a background note.
| 2026-08-27 | intelligence | `distillLearning()`'s EWC forgetting-gate called `getPenalty([oldConf],[newConf])`, collapsing to 1 of 384 Fisher dims β€” wired in `computeConfidencePenalty`/`updateFisherFromConfidences` (already implemented, 0 prior callers). **Post-review correction (ruvnet):** original hypothesis overclaimed per-pattern discrimination; `computeConfidencePenalty` takes no embedding, so the real fix is "shared full-Fisher-average signal instead of one arbitrary dimension," not per-pattern weighting β€” hypothesis/gist/tests amended same night, new acceptance test added proving no per-pattern discrimination | #3109 | #3110 | yes | ACCEPT-scoped (narrowed post-review) | baseline: regression test fails (methods never called); candidate: passes. 153/153 tests (+5 new incl. acceptance test), tsc clean | 0df6bc4f... | MERGED 2026-09-14 (verified: PR state=MERGED, mergedAt=2026-09-14T20:46:20Z) |
| 2026-08-28 | memory | `HybridBackend.queryHybrid()`'s dead `weights` field (computed, never consumed by combineUnion) replaced with real weighted-RRF fusion (`combineWeighted`), matching Qdrant/Weaviate/Azure AI Search's weighted-RRF pattern | #3118 | #3119 | yes | ACCEPT | 461/462 suite green (1 pre-existing unrelated env failure, confirmed identical on baseline); weight-sensitivity test fails on baseline, passes on candidate (git-stash isolated); post-review: added finite/non-negative weight validation + deterministic id-based tie-break | 47e2f669... | MERGED 2026-09-14 (verified via main's own commit fde5b3b) |
| 2026-08-29 | swarm | `TopologyManager.rebalanceHybrid()`'s worker-mesh loop added edges one-directionally (worker→target only, never reciprocated) — same function's coordinator loop and all 3 sibling rebalance*() methods do this correctly; flagged live by 08-24's own night, fixed tonight | #3122 | #3123 | yes | ACCEPT | deterministic (Math.random mocked) regression test: baseline missing reciprocal edge, candidate symmetric; full package 221/221 tests, 0 regressions; independent adversarial critic CONFIRMED | 9c1e9cd4... | rebased onto main (post #3086/#3094/#3103/#3110/#3119 merges), awaiting fresh CI as of 2026-09-14 |
-| 2026-08-30 | performance | unwired diskann-backend.ts removed + stale 150x-12,500x HNSW claim in generated CLAUDE.md corrected | #3129 | #3130 | yes | ACCEPT | see issue #3129 | n/a (backfill) | unmerged draft as of 09-05 |
+| 2026-08-30 | performance | `diskann-backend.ts` (375 lines, zero call sites anywhere in the monorepo, `@ruvector/diskann` not a dependency) removed; `claudemd-generator.ts`'s generated CLAUDE.md corrected from the unreproduced "150x-12,500x"/DiskANN claim to point at the reproducible `benchmark-intelligence.mjs --only=hnsw` command instead of a hardcoded multiplier | #3129 | #3130 | yes | ACCEPT | baseline-fails/candidate-passes test confirmed via git-stash; full suite byte-identical 106-failure list baseline vs candidate (pre-existing env gap); live re-run of benchmark-intelligence.mjs found recall@10 0.9867(N=5k)/0.9233(N=20k) β€” itself disagreeing with the "~0.99" doc figure, reported not papered over; reviewer REJECTed v1 (CI red, 11-row ledger backfill, ambiguous cited number) β€” all 3 addressed same night | 6f56440a... | rebased onto main (post #3086/#3094/#3103/#3110/#3119/#3123 merges), awaiting fresh CI as of 2026-09-14 |
| 2026-08-31 | security | MCP governance policy (.harness/mcp-policy.json) was never enforced at runtime β€” opt-in enforcement wired in | #3138 | #3139 | yes | ACCEPT | see issue #3138 | n/a (backfill) | unmerged draft as of 09-05 |
| 2026-09-01 | security | maxToolCallsPerTurn was a session-lifetime cumulative cap that never reset β€” sliding-window reset implemented | #3151 | #3152 | yes | ACCEPT | see issue #3151 | n/a (backfill) | unmerged draft as of 09-05 |
| 2026-09-02 | intelligence | LearningBridge.consolidate() was reward-blind β€” fixed to weight consolidation by trajectory reward | #3159 | #3160 | yes | ACCEPT | see issue #3159 | n/a (backfill) | unmerged draft as of 09-05 |
diff --git a/docs/dream-cycle/dream-gist-2026-08-30.md b/docs/dream-cycle/dream-gist-2026-08-30.md
new file mode 100644
index 0000000000..843a479674
--- /dev/null
+++ b/docs/dream-cycle/dream-gist-2026-08-30.md
@@ -0,0 +1,91 @@
+# Performance SOTA Report β€” 2026-08-30
+
+TL;DR: Tonight's performance deep-dive found that the audit-corrected HNSW figure ("~1.9x-4.7x vs brute force", `docs/reviews/intelligence-system-audit-2026-05-29.md`) never propagated into the CLI's own generated project docs — `ruflo init` was still writing an unsubstantiated "HNSW/DiskANN: 150x-12,500x faster search" claim into every new user's CLAUDE.md, alongside a reference to DiskANN as an active backend. Investigation found `diskann-backend.ts` (375 lines, a full 3-tier DiskANN→HNSW→brute-force fallback chain) has zero call sites anywhere in the monorepo and `@ruvector/diskann` isn't even a listed dependency — it never shipped. Fixed both in `claudemd-generator.ts` (2026-08-25/08-15's established "found-via-code-read, small-diff, deterministic-test" pattern), with a regression test pinning the corrected figure across every template. The same debunked "150x-12,500x" string also still appears live in ~14 other CLI source locations (help text, MCP tool metadata, command output) and a "2.49x-7.47x" Flash Attention figure with the identical problem in ~9 more — both **out of scope tonight** (see Recommended Next Steps) to keep this patch small and reviewable.
+
+**Post-review update (same night):** the repo owner reviewed the first version of this candidate and rejected it on three grounds, all addressed below and re-evaluated: (1) CI red on this exact head from a repo-wide `@claude-flow/[email protected]` install failure β€” confirmed pre-existing on `main`, partially fixed here by porting the accepted `v3/pnpm-lock.yaml` fix from open PR #3104, with the remaining root-npm side correctly left to maintainer judgment per issue #3095's own text; (2) this branch had accumulated an 11-row ledger backfill (2026-08-20 through 08-29) instead of exactly one row for this run β€” trimmed to one row, ledger repair deferred to its own effort; (3) the replacement HNSW claim itself cited a doc (`~1.9x-4.7x, recall@10 ~0.99`) that self-contradicts elsewhere (`~0.9` vs `~0.99`) β€” replaced with a live re-run of `scripts/benchmark-intelligence.mjs` on this exact commit (see Evaluation) and the generated docs now point at the reproducible command instead of a hardcoded multiplier, so this can't drift stale again.
+
+## What's New in 2026
+
+| Finding | Source | Confidence |
+|---|---|---|
+| CHAT: constraint-aware, feasibility-boundary HNSW parameter tuning (M/efConstruction/efSearch) under recall/latency/memory constraints | arXiv:2607.04630 (2026) | B (preprint, methodology directly applicable) |
+| RaBitQ 1-bit quantization with theoretical error bound, SIMD/bitwise distance estimation | Gao & Long, SIGMOD/PACMMOD 2024 (dl.acm.org/doi/10.1145/3654970) | A (peer-reviewed; this is the algorithm Ruflo's `rabitq-index.ts` already wraps) |
+| SegPQ β€” lossless PQ codebook compression for memory-constrained ANN | VLDB 2025 (dl.acm.org/doi/10.14778/3749646.3749650) | A |
+| CS-PQ β€” cache-friendly SIMD product quantization for ANNS construction | arXiv:2605.25521 (2026) | B β€” directly relevant: Ruflo's PQ `kMeans`/`encode` use naive nested loops, no SIMD/cache-blocking |
+| Correlation-aware/surrogate-reward contextual bandits for LLM routing | arXiv:2607.09015 (2026) | B β€” matches a real gap: Ruflo's `model-router.ts` bandit reward is a flat `{model,outcome}` constant table, blind to the router's own continuous `predictedQuality` signal |
+| Multi-agent KV-cache sharing (PolyKV, QKVShare, TokenDance, DroidSpeak) | arXiv:2604.24971, 2605.03884, 2604.03143, 2411.02820 | B/C β€” **not applicable to Ruflo**: these all assume the caller hosts the model/attention state; Ruflo orchestrates hosted-API calls per agent and has no local KV cache to share |
+| CLEAR framework: 37% average gap between lab-benchmark and production performance for multi-agent systems | Cited via 2026 framework survey | B (secondhand) β€” a caution worth internalizing for Ruflo's own benchmark claims |
+
+## Ruflo Current Capability
+
+Vector search in this repo is fragmented across **three non-interoperating stacks**, only one of which is actually benchmarked end-to-end: `@claude-flow/memory`'s pure-JS `HNSWIndex`/`AgentDBAdapter` (the package's own documented public API, never recall-tested against brute force); `@claude-flow/cli/src/ruvector/vector-db.ts`'s NAPI-accelerated `VectorDb` (the *only* path `scripts/benchmark-intelligence.mjs` measures β€” source of the real "~1.9x-4.7x" number); and `memory-initializer.ts`'s own third, independent `HNSWIndex` interface backed by `@ruvector/core` with a correctly-split `efConstruction`/`efSearch` schema. A fourth, `rabitq-index.ts`'s 32x-compression path, is structurally unreachable from `@claude-flow/memory` consumers. `@claude-flow/memory`'s own `HNSWIndex.search()` has no `efSearch` field distinct from `efConstruction` at all β€” every call through the documented `AgentDBAdapter.semanticSearch()` API runs at `ef = efConstruction = 200` with no speed/recall tradeoff knob. (Note: 2026-08-15's PR #3034 tried adding exactly this field with a flat `efSearch=50` default and was **REJECTED** β€” recall dropped to 0.8767 at N=8000, breaching the 0.90 floor. Tonight's research treats that as still-standing evidence against a flat low default, not reopened without a scale-adaptive redesign β€” see Recommended Next Steps.)
+
+Separately, `diskann-backend.ts` implements a complete `diskann β†’ hnsw β†’ cosine-js` fallback chain, including its own benchmark utility, but is imported by **nothing** β€” not `ruvector/index.ts`'s barrel export, not any command, not any test. `claudemd-generator.ts`'s generated CLAUDE.md told every new project to "Use HNSW/DiskANN for vector search" and claimed "HNSW/DiskANN: 150x-12,500x faster search" β€” a number this repo's own May 2026 audit explicitly could not reproduce and corrected to ~1.9x-4.7x in its own source-of-truth docs, five months before tonight.
+
+## Competitor Comparison
+
+| Framework | 2025-2026 performance work | Grade |
+|---|---|---|
+| Qdrant | GPU-accelerated HNSW build; 1.5/2-bit + asymmetric quantization beyond int8; inline-storage quantized-in-graph layout | B |
+| Weaviate | Native BM25+vector hybrid; "Search Mode" test-time-compute scaling across 12 benchmarks (BEIR/LoTTe/BRIGHT/...); SQ8 ~4x memory, 1-2% recall loss | B |
+| Milvus | GPU CAGRA (NVIDIA cuVS): ~50x search throughput, 12.5x better time-to-cost at top100@98% recall | A (methodology + hardware stated) |
+| LanceDB | RaBitQ GA (1-bit + centroid-only routing, O(D log D) query prep) targeting 10B-vector scale | B |
+| Vespa | 8.5-12.9x throughput/core vs Elasticsearch | B (Jan 2025, aging) |
+| LangGraph / AutoGen / CrewAI / OpenAI Agents SDK | None ship a built-in cost/latency model router or cross-agent KV-cache sharing β€” LangGraph/OpenAI SDK leave caching to the model provider by design, not by omission | B |
+
+**Verdict**: every major vector DB now ships GPU-accelerated indexing that Ruflo lacks β€” a genuine, open, unhidden gap. Where Ruflo differentiates (3-tier cost/latency routing, SONA) is unclaimed territory for all four agent-framework competitors surveyed, deliberately so in two cases.
+
+## Hypothesis
+
+> Given a new project created via `ruflo init` (default or `--wizard`), when the generated CLAUDE.md's performance/intelligence sections are corrected to drop the DiskANN claim (its only integration, `diskann-backend.ts`, has zero call sites in the monorepo and `@ruvector/diskann` is not a dependency) and replace "150x-12,500x faster search" with a claim Ruflo can actually substantiate, then generated project documentation states only claims Ruflo can substantiate, subject to: (1) no other generated-doc section changes; (2) zero import/build errors anywhere in the monorepo from removing `diskann-backend.ts`; (3) existing test suite green; (4) $0 cost. Frozen before evaluation; not modified after.
+>
+> **Revised post-review**: "a claim Ruflo can actually substantiate" was originally instantiated as a copied number ("~1.9x-4.7x, recall@10 ~0.99") from `docs/reviews/intelligence-system-audit-2026-05-29.md`. The reviewer correctly flagged that this doc itself contains both "~0.9" and "~0.99" recall language β€” an unresolved ambiguity, not a citable fact. The hypothesis's *intent* (state only what Ruflo can substantiate) is unchanged; its *instantiation* is revised to point the generated docs at the live reproducible command (`node scripts/benchmark-intelligence.mjs --only=hnsw`) rather than any hardcoded number, so the claim can never re-drift stale the way "150x-12,500x" did.
+
+## Evaluation
+
+**evaluated: accepted.** Real evaluator: `vitest run` + `tsc --noEmit`, deterministic, $0. Baseline reproduced via `git stash` (source+test), confirmed the new test fails exactly as predicted (both stale strings present in the 'performance'/'full' template output); candidate (restored) passes 17/17 in the touched describe block. Full package suite: candidate and a controlled baseline re-run produced **byte-identical** 106-line failure lists (pre-existing, environmental β€” missing native `@ruvector/*-wasm` modules in this sandbox, same documented class as prior nights); an initial pair of ad hoc full-suite runs showed differing raw counts (104 vs 85) before the controlled comparison β€” disclosed as suite-level flakiness unrelated to this diff, not hidden. `tsc --noEmit`: pre-existing errors are all unbuilt-sibling-package (`@claude-flow/memory`, `@claude-flow/neural`, `@claude-flow/swarm`) resolution failures, none referencing either touched file.
+
+**Reproducible benchmark receipt (added post-review, per reviewer's "one unambiguous fixture and command" ask).** Built `v3/@claude-flow/cli` (`npm run build`; the same pre-existing unbuilt-sibling-package `tsc` errors above, dist still emits) and ran the exact command now referenced in the generated docs, twice, on this session's host:
+
+```
+node scripts/benchmark-intelligence.mjs --only=hnsw --sizes 5000,20000 --queries 30
+```
+
+| N | run 1 speedup | run 2 speedup | recall@10 (both runs, identical) |
+|--:|--:|--:|--:|
+| 5000 | 3.62x | 2.85x | 0.9867 |
+| 20000 | 6.77x | 7.58x | 0.9233 |
+
+`recall@10` is exactly reproducible across runs (seeded RNG for vector generation); `speedup` varies with host load (wall-clock timing), always >1x in both runs. This is the reviewer's requested "unambiguous fixture" β€” **and it disagrees with the doc figure it was meant to confirm**: recall@10 at N=20000 measured 0.9233, not "~0.99" as both the audit doc and this candidate's first version claimed. That is itself new evidence the "~1.9x-4.7x/recall~0.99" figure needs its own re-verification night (see Recommended Next Steps) β€” not something to quietly paper over by picking whichever cited number looked closest. The shipped fix no longer hardcodes any of these numbers into generated docs; it names the reproduction command instead.
+
+**Independent adversarial critic** (fresh session, no authoring context): re-verified zero call sites for every `diskann-backend.ts` export across the *whole* repo (not just `cli/src`), including ruling out a false-positive `'diskann'` string in an unrelated backend-type enum in `@claude-flow/plugins`; independently re-read `docs/reviews/intelligence-system-audit-2026-05-29.md` and confirmed "~1.9x-4.7x, recall@10 ~0.99" is the correct current figure (noting one pre-existing, not-introduced-by-this-diff wrinkle: the audit doc itself says "recall ~0.9" in one older passage vs. "~0.99" elsewhere); reran `tsc --noEmit | grep diskann` (empty) and the new test in isolation (genuinely fails pre-fix, passes post-fix β€” not reward-hacked). **Verdict: CONFIRMED**, with one disclosure gap: the *single most user-visible* instance of this exact bug β€” `v3/@claude-flow/cli/src/commands/memory.ts:1153`'s live `output.printInfo('V3 Performance: 150x-12,500x faster search...')`, printed at runtime, not just in generated docs β€” is in the same package this candidate touched and was not fixed here. Correct call to keep out of scope (bundling it would have meant fixing an open-ended, ~15-location, cross-package sweep in one night), but it means tonight's fix, while real, is the least-visible instance of the underlying problem, not the most-visible one. Folded into Recommended Next Steps #3 below rather than silently left out.
+
+## CI Remediation (post-review)
+
+Both commits on this branch failed nearly every CI check at the shared `npm ci`/`pnpm install` step β€” confirmed pre-existing on `main` at this PR's base commit before this branch existed (identical `ETARGET` on `@claude-flow/[email protected]`, which the npm registry has never published; latest published is `3.0.0-alpha.9`). Two already-tracked issues cover this: #3101 (pnpm-lockfile side, has an open fix PR #3104) and #3095 (root-npm side, no patch proposed by design β€” "dependency/release surfaces require maintainer validation"). Ported #3104's one-line `v3/pnpm-lock.yaml` fix into this branch (validated: `corepack pnpm install --frozen-lockfile --lockfile-only` now succeeds) β€” this resolves the v3-pnpm-driven jobs (Type Check V3, Test V3 Packages, etc.). The root-npm-side jobs (Integration Test Setup, Setup Verification, Audit root, CI/CD Pipeline, most smoke tests) remain blocked on #3095, which explicitly defers to a maintainer decision (publish `alpha.10` vs. revert the pin) rather than a contributor patch β€” not touched here for that reason, not because it was missed.
+
+## Darwin Results
+
+Skipped β€” scope mismatch. This is a documentation-accuracy/dead-code-removal fix with no continuous/categorical parameter; `darwin --execute`'s real interface evolves routing/topology/prompt/memory/tool/tier/context/coordination genome parameters against an LLM-scored bench corpus. Same skip class as 7 of the last 8 dream-cycle nights.
+
+## SOTA Proof & Witness
+
+| Field | Value |
+|---|---|
+| Session commit | `d33ef4bf8ab27a8f9ef08352c9c293b53312a861` |
+| Gist SHA-256 (pre-witness content, i.e. this file before this table was filled in, post-review revision) | `2a02686c4b05dc12e8c8b6e2a539f8605dde527ab04a4a72a09602f56e3ac1ab` |
+| Witness stamp | `6f56440a68f4ad0cd43acf45d8b486b263c6106898438a373a39f8914628b672` |
+
+Verifier procedure: fetch `docs/dream-cycle/dream-gist-2026-08-30.md` from the `dream/2026-08-30-performance` branch, strip this table back to the placeholder line, SHA-256 the result, concatenate with the session commit above, SHA-256 again β€” result must equal the witness stamp. (Superseded the pre-review witness `3c98578c...`; this is the current one after addressing the reviewer's REJECT.)
+
+Verifier procedure: fetch `docs/dream-cycle/dream-gist-2026-08-30.md` from the `dream/2026-08-30-performance` branch, strip this table back to the placeholder line ("See issue and PR for the full witness table..."), SHA-256 the result, concatenate with the session commit above, SHA-256 again β€” result must equal the witness stamp.
+
+## Recommended Next Steps
+
+1. **Merge this candidate** (human review required) β€” removes 375 lines of unreachable, misleading code, closes a real gap between this repo's own docs and what it tells every new user, and now points at a live command instead of a re-driftable number.
+2. **Dedicated benchmark re-verification night**: this session's own reproducible run found `~1.9x-4.7x/recall~0.99` does not hold uniformly (0.9233 recall@10 at N=20000) β€” the audit doc (`docs/reviews/intelligence-system-audit-2026-05-29.md`) and root `CLAUDE.md`/`v3/CLAUDE.md` need a fresh, single, unambiguous re-run and correction, not another copied citation.
+3. **The same "150x-12,500x" string is still live in ~14 other CLI source locations** (`src/index.ts` `--help` output, `mcp-tools/{memory,hooks}-tools.ts`, `init/executor.ts`'s generated CAPABILITIES.md, `commands/{ruvector,embeddings,hooks,memory,agent,performance}.ts`) and a structurally identical "2.49x-7.47x" Flash Attention figure appears in ~9 more locations, plus ~15 `.claude/skills`/`.claude/agents` SKILL.md files. **Highest-priority single instance**: `commands/memory.ts:1153`'s `output.printInfo('V3 Performance: 150x-12,500x faster search...')` is live runtime CLI output, more user-visible than what this candidate fixed. Worth a dedicated, ideally scripted, sweep night.
+4. **Root-npm `@claude-flow/[email protected]` pin** (issue #3095) needs a maintainer decision (publish `.10` or revert to `.9`) β€” blocks root-npm CI jobs repo-wide, not just this PR, and only touched here via the safe, already-open, pnpm-side fix (#3104).
+5. Re-open the `@claude-flow/memory` `efSearch`/`efConstruction` decoupling question **only** with a scale-adaptive default (per CHAT, arXiv:2607.04630) β€” not a flat low default, which is exactly what 2026-08-15's PR #3034 tested and got rejected for.
+6. `Quantizer.productQuantizeDistance()` is still unwired into `HNSWIndex.distance()` five research-nights after 2026-08-25 first reported it β€” cheap, high-confidence, still open.
+7. Consider a graded/surrogate reward signal for `model-router.ts`'s bandit using its own `predictedQuality` output (arXiv:2607.09015) β€” medium risk given ADR-148/149/150 already build on the current reward shape; recommend offline replay against `.swarm/model-router-state.json` before any live change.
diff --git a/v3/@claude-flow/cli/__tests__/commands-deep.test.ts b/v3/@claude-flow/cli/__tests__/commands-deep.test.ts
index 48e56cfd09..3934aac52f 100644
--- a/v3/@claude-flow/cli/__tests__/commands-deep.test.ts
+++ b/v3/@claude-flow/cli/__tests__/commands-deep.test.ts
@@ -1590,6 +1590,30 @@ describe('Init System', () => {
}
});
+ it('should not assert the debunked 150x-12,500x HNSW figure or an unreachable DiskANN capability, and should point at the reproducible benchmark instead of a hardcoded multiplier (dream/2026-08-30)', () => {
+ // The 150x-12,500x number was measured to be unreproducible (see
+ // docs/reviews/intelligence-system-audit-2026-05-29.md) and
+ // diskann-backend.ts β€” the only DiskANN integration in this repo β€”
+ // had zero call sites anywhere in the monorepo before its removal
+ // tonight. A live re-run of scripts/benchmark-intelligence.mjs during
+ // this same dream-cycle night found recall@10 0.9867 at N=5000 but
+ // only 0.9233 at N=20000 β€” i.e. even the previously "corrected"
+ // ~1.9x-4.7x/recall~0.99 figure doesn't hold uniformly, so the fix
+ // points at the reproducible command instead of hardcoding a new
+ // multiplier that would just go stale again. Only the 'performance'
+ // and 'full' templates render performanceSection()/intelligenceSystem()
+ // at all, so check every template rather than just the default.
+ for (const template of CLAUDE_MD_TEMPLATES) {
+ const md = generateClaudeMd(DEFAULT_INIT_OPTIONS, template.name);
+ expect(md).not.toContain('150x-12,500x');
+ expect(md).not.toContain('DiskANN');
+ }
+ const perfMd = generateClaudeMd(DEFAULT_INIT_OPTIONS, 'performance');
+ expect(perfMd).toContain('benchmark-intelligence.mjs --only=hnsw');
+ const fullMd = generateClaudeMd(DEFAULT_INIT_OPTIONS, 'full');
+ expect(fullMd).toContain('benchmark-intelligence.mjs --only=hnsw');
+ });
+
it('should describe anti-drift swarm topology', () => {
const md = generateClaudeMd(DEFAULT_INIT_OPTIONS, 'standard');
expect(md).toContain('anti-drift');
diff --git a/v3/@claude-flow/cli/src/init/claudemd-generator.ts b/v3/@claude-flow/cli/src/init/claudemd-generator.ts
index 2fc6250fcc..f35bd99358 100644
--- a/v3/@claude-flow/cli/src/init/claudemd-generator.ts
+++ b/v3/@claude-flow/cli/src/init/claudemd-generator.ts
@@ -272,7 +272,7 @@ function performanceSection(): string {
- Always benchmark before AND after optimization
- Always profile before optimizing β€” never guess bottlenecks
-- Use HNSW/DiskANN for vector search, Int8 quantization for memory reduction
+- Use HNSW for vector search, Int8 quantization for memory reduction β€” measure the actual speedup/recall on your own corpus via \`node scripts/benchmark-intelligence.mjs --only=hnsw\` rather than trusting a hardcoded multiplier (recall@10 varies by corpus size β€” 0.99 at N=5k, 0.92 at N=20k on the reference host, dream-cycle 2026-08-30)
\`\`\`bash
npx @claude-flow/cli@latest performance benchmark --suite all
@@ -309,7 +309,7 @@ function intelligenceSystem(): string {
Pipeline: **RETRIEVE** (vector search) β†’ **JUDGE** (success/failure) β†’ **DISTILL** (extract patterns) β†’ **CONSOLIDATE** (persist)
- **ONNX Embeddings**: all-MiniLM-L6-v2, 384-dim
-- **HNSW/DiskANN**: 150x-12,500x faster search
+- **HNSW**: faster than brute-force search, magnitude scales with corpus size β€” reproduce with \`node scripts/benchmark-intelligence.mjs --only=hnsw\` rather than a hardcoded multiplier
- **SONA**: Sub-millisecond pattern adaptation
- **Claude Bridge**: Auto-imports \`~/.claude/projects/*/memory/*.md\` into AgentDB`;
}
diff --git a/v3/@claude-flow/cli/src/ruvector/diskann-backend.ts b/v3/@claude-flow/cli/src/ruvector/diskann-backend.ts
deleted file mode 100644
index 83cbf67642..0000000000
--- a/v3/@claude-flow/cli/src/ruvector/diskann-backend.ts
+++ /dev/null
@@ -1,375 +0,0 @@
-/**
- * DiskANN Vector Search Backend
- *
- * SSD-friendly approximate nearest neighbor search using Vamana graph.
- * Falls back gracefully to HNSW (@ruvector/router VectorDb) or
- * pure-JS cosine similarity when DiskANN is unavailable.
- *
- * @module v3/cli/ruvector/diskann-backend
- */
-
-// ===== Types =====
-
-export interface DiskAnnConfig {
- dim: number;
- maxDegree?: number;
- buildBeam?: number;
- searchBeam?: number;
- alpha?: number;
- pqSubspaces?: number;
- storagePath?: string;
-}
-
-export interface SearchResult {
- id: string;
- distance: number;
- score: number; // Converted to similarity (1 / (1 + distance))
-}
-
-export type VectorBackend = 'diskann' | 'hnsw' | 'cosine-js';
-
-// ===== Lazy loading =====
-
-let diskannInstance: any = null;
-let diskannAvailable: boolean | null = null;
-let activeBackend: VectorBackend = 'cosine-js';
-
-/**
- * Check if @ruvector/diskann is available
- */
-export async function isDiskAnnAvailable(): Promise<boolean> {
- if (diskannAvailable !== null) return diskannAvailable;
- try {
- const { createRequire } = await import('module');
- const require2 = createRequire(import.meta.url);
- const mod = require2('@ruvector/diskann');
- diskannAvailable = typeof mod.DiskAnn === 'function';
- return diskannAvailable;
- } catch {
- diskannAvailable = false;
- return false;
- }
-}
-
-/**
- * Create or get a DiskANN index instance
- */
-export async function getDiskAnnIndex(config: DiskAnnConfig): Promise<{
- index: any;
- backend: VectorBackend;
-}> {
- if (diskannInstance) return { index: diskannInstance, backend: activeBackend };
-
- // Try DiskANN first
- if (await isDiskAnnAvailable()) {
- try {
- const { createRequire } = await import('module');
- const require2 = createRequire(import.meta.url);
- const { DiskAnn } = require2('@ruvector/diskann');
-
- const index = new DiskAnn({
- dim: config.dim,
- maxDegree: config.maxDegree ?? 64,
- buildBeam: config.buildBeam ?? 128,
- searchBeam: config.searchBeam ?? 64,
- alpha: config.alpha ?? 1.2,
- pqSubspaces: config.pqSubspaces ?? 0,
- storagePath: config.storagePath,
- });
-
- diskannInstance = index;
- activeBackend = 'diskann';
- return { index, backend: 'diskann' };
- } catch {
- // Fall through
- }
- }
-
- // Try HNSW (@ruvector/router VectorDb) as fallback
- try {
- const { createRequire } = await import('module');
- const require2 = createRequire(import.meta.url);
- const router = require2('@ruvector/router');
- if (router.VectorDb && router.DistanceMetric) {
- const index = new router.VectorDb({
- dimensions: config.dim,
- distanceMetric: router.DistanceMetric.Cosine,
- hnswM: 16,
- hnswEfConstruction: 200,
- hnswEfSearch: 100,
- });
- diskannInstance = index;
- activeBackend = 'hnsw';
- return { index, backend: 'hnsw' };
- }
- } catch {
- // Fall through
- }
-
- // Pure JS fallback
- const jsIndex = createJsFallbackIndex(config.dim);
- diskannInstance = jsIndex;
- activeBackend = 'cosine-js';
- return { index: jsIndex, backend: 'cosine-js' };
-}
-
-/**
- * Get the active backend name
- */
-export function getActiveBackend(): VectorBackend {
- return activeBackend;
-}
-
-/**
- * Reset the index (for testing)
- */
-export function resetIndex(): void {
- diskannInstance = null;
- diskannAvailable = null;
- activeBackend = 'cosine-js';
-}
-
-// ===== Unified search interface =====
-
-/**
- * Insert a vector into the active backend
- */
-export async function insertVector(
- id: string,
- vector: Float32Array,
- config: DiskAnnConfig = { dim: 384 }
-): Promise<{ backend: VectorBackend }> {
- const { index, backend } = await getDiskAnnIndex(config);
-
- if (backend === 'diskann') {
- index.insert(id, vector);
- } else if (backend === 'hnsw') {
- index.insert(id, vector);
- } else {
- index.insert(id, vector);
- }
-
- return { backend };
-}
-
-/**
- * Build the index (required for DiskANN before search)
- */
-export async function buildIndex(config: DiskAnnConfig = { dim: 384 }): Promise<void> {
- const { index, backend } = await getDiskAnnIndex(config);
- if (backend === 'diskann' && typeof index.build === 'function') {
- index.build();
- }
- // HNSW and JS fallback don't need explicit build
-}
-
-/**
- * Search for k nearest neighbors
- */
-export async function searchVectors(
- query: Float32Array,
- k: number,
- config: DiskAnnConfig = { dim: 384 }
-): Promise<SearchResult[]> {
- const { index, backend } = await getDiskAnnIndex(config);
-
- if (backend === 'diskann') {
- const results = index.search(query, k);
- return results.map((r: any) => ({
- id: r.id,
- distance: r.distance,
- score: 1 / (1 + r.distance), // Convert L2 distance to similarity
- }));
- }
-
- if (backend === 'hnsw') {
- const results = index.search(query, k);
- return results.map((r: any) => ({
- id: r.id,
- distance: r.score, // VectorDb returns distance as 'score'
- score: 1 / (1 + r.score),
- }));
- }
-
- // JS fallback
- return index.search(query, k);
-}
-
-// ===== Pure JS fallback =====
-
-function createJsFallbackIndex(dim: number) {
- const vectors = new Map<string, Float32Array>();
-
- return {
- insert(id: string, vector: Float32Array): void {
- vectors.set(id, new Float32Array(vector));
- },
- search(query: Float32Array, k: number): SearchResult[] {
- const results: SearchResult[] = [];
- for (const [id, vec] of vectors) {
- let dot = 0, normQ = 0, normV = 0;
- for (let i = 0; i < dim; i++) {
- dot += query[i] * vec[i];
- normQ += query[i] * query[i];
- normV += vec[i] * vec[i];
- }
- const cosine = dot / (Math.sqrt(normQ) * Math.sqrt(normV) || 1);
- const distance = 1 - cosine;
- results.push({ id, distance, score: cosine });
- }
- return results.sort((a, b) => a.distance - b.distance).slice(0, k);
- },
- count(): number { return vectors.size; },
- delete(id: string): boolean { return vectors.delete(id); },
- build(): void { /* no-op */ },
- };
-}
-
-// ===== Benchmark utility =====
-
-export interface BenchmarkResult {
- backend: VectorBackend;
- dim: number;
- vectorCount: number;
- insertTimeMs: number;
- buildTimeMs: number;
- searchTimeMs: number;
- searchesPerSecond: number;
- recall: number; // vs brute force
- memoryMB: number;
-}
-
-/**
- * Run a benchmark comparing available backends
- */
-export async function benchmark(opts: {
- dim?: number;
- vectorCount?: number;
- k?: number;
- queries?: number;
-} = {}): Promise<BenchmarkResult[]> {
- const dim = opts.dim ?? 384;
- const n = opts.vectorCount ?? 1000;
- const k = opts.k ?? 10;
- const queryCount = opts.queries ?? 100;
- const results: BenchmarkResult[] = [];
-
- // Generate random vectors
- const vectors: Array<[string, Float32Array]> = [];
- for (let i = 0; i < n; i++) {
- const v = new Float32Array(dim);
- for (let j = 0; j < dim; j++) v[j] = Math.random() * 2 - 1;
- // Normalize
- let norm = 0;
- for (let j = 0; j < dim; j++) norm += v[j] * v[j];
- norm = Math.sqrt(norm);
- for (let j = 0; j < dim; j++) v[j] /= norm;
- vectors.push([`vec-${i}`, v]);
- }
-
- // Generate query vectors
- const queries: Float32Array[] = [];
- for (let i = 0; i < queryCount; i++) {
- const q = new Float32Array(dim);
- for (let j = 0; j < dim; j++) q[j] = Math.random() * 2 - 1;
- let norm = 0;
- for (let j = 0; j < dim; j++) norm += q[j] * q[j];
- norm = Math.sqrt(norm);
- for (let j = 0; j < dim; j++) q[j] /= norm;
- queries.push(q);
- }
-
- // Brute force ground truth
- function bruteForceSearch(query: Float32Array): string[] {
- const scores: Array<{ id: string; dist: number }> = [];
- for (const [id, vec] of vectors) {
- let dist = 0;
- for (let j = 0; j < dim; j++) {
- const d = query[j] - vec[j];
- dist += d * d;
- }
- scores.push({ id, dist });
- }
- scores.sort((a, b) => a.dist - b.dist);
- return scores.slice(0, k).map(s => s.id);
- }
-
- const groundTruth = queries.map(q => bruteForceSearch(q));
-
- // Test each available backend
- for (const backendName of ['diskann', 'hnsw', 'cosine-js'] as VectorBackend[]) {
- resetIndex();
-
- try {
- let index: any;
- const memBefore = process.memoryUsage().heapUsed;
-
- if (backendName === 'diskann') {
- if (!(await isDiskAnnAvailable())) continue;
- const { createRequire } = await import('module');
- const require2 = createRequire(import.meta.url);
- const { DiskAnn } = require2('@ruvector/diskann');
- index = new DiskAnn({ dim, maxDegree: 64, buildBeam: 128, searchBeam: 64 });
- } else if (backendName === 'hnsw') {
- try {
- const { createRequire } = await import('module');
- const require2 = createRequire(import.meta.url);
- const router = require2('@ruvector/router');
- if (!router.VectorDb) continue;
- index = new router.VectorDb({ dimensions: dim, distanceMetric: router.DistanceMetric.Cosine });
- } catch { continue; }
- } else {
- index = createJsFallbackIndex(dim);
- }
-
- // Insert
- const insertStart = performance.now();
- for (const [id, vec] of vectors) {
- index.insert(id, vec);
- }
- const insertTime = performance.now() - insertStart;
-
- // Build
- const buildStart = performance.now();
- if (typeof index.build === 'function') index.build();
- const buildTime = performance.now() - buildStart;
-
- // Search
- const searchStart = performance.now();
- const searchResults: string[][] = [];
- for (const q of queries) {
- const r = index.search(q, k);
- searchResults.push(r.map((x: any) => x.id));
- }
- const searchTime = performance.now() - searchStart;
-
- // Recall vs ground truth
- let totalRecall = 0;
- for (let i = 0; i < queryCount; i++) {
- const truth = new Set(groundTruth[i]);
- const found = searchResults[i].filter(id => truth.has(id)).length;
- totalRecall += found / k;
- }
- const recall = totalRecall / queryCount;
-
- const memAfter = process.memoryUsage().heapUsed;
-
- results.push({
- backend: backendName,
- dim,
- vectorCount: n,
- insertTimeMs: Math.round(insertTime * 100) / 100,
- buildTimeMs: Math.round(buildTime * 100) / 100,
- searchTimeMs: Math.round(searchTime * 100) / 100,
- searchesPerSecond: Math.round(queryCount / (searchTime / 1000)),
- recall: Math.round(recall * 1000) / 1000,
- memoryMB: Math.round((memAfter - memBefore) / 1024 / 1024 * 100) / 100,
- });
- } catch {
- // Backend failed, skip
- }
- }
-
- resetIndex();
- return results;
-}