Skip to content

armgit / ruflo

Public Imported from GitHub
0

🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated

dream(intelligence): #3159 LearningBridge.consolidate() reward-blind fix (evaluated, ACCEPT) (#3160)

rUv755b6720c52a
diff --git a/docs/dream-cycle/LEDGER.md b/docs/dream-cycle/LEDGER.md
index cece16a793..3cd198a309 100644
--- a/docs/dream-cycle/LEDGER.md
+++ b/docs/dream-cycle/LEDGER.md
@@ -187,3 +187,4 @@ as a first-class, high-priority candidate, not a background note.
| 2026-09-09 | swarm | AgentPool.performHealthChecks() stamped lastHeartbeat=now on every "still healthy" tick, and updateAgentHeartbeat() (the only other legitimate way to advance it) has zero callers anywhere in v3/ β€” so the check's own tick was the sole thing keeping every pooled agent's heartbeat fresh, meaning timeSinceLastActivity could never exceed one interval and replaceUnhealthyAgent() could never fire for any agent, in any pool, ever; removed the self-stamp | #3242 | #3243 | yes | ACCEPT-scoped | 223/223 tests (+3 new), baseline-fails-2-of-3/candidate-passes-3-of-3 via stash isolation; adversarial critic confirmed independently but disclosed @claude-flow/swarm has no reachable path through the shipped @claude-flow/cli today (blast radius is the swarm package's own UnifiedSwarmCoordinator, not an active CLI hot path) | 008c96a0... | 08-24..09-02 backlog unchanged (see note above); #3169/#3184/#3221 merged since 09-05 |
| 2026-09-01 | security | `maxToolCallsPerTurn` was session-lifetime cumulative, never reset (disclosed bug in #3139); wall-clock sliding window fix, new `turnWindowMs` policy field (follow-up to #3139, stacked PR) | #3151 | #3152 | yes | ACCEPT | 30/30 new-suite tests, 52/56 broader MCP sweep (4 pre-existing unbuilt-sibling-package env failures, unrelated); independent critic CONFIRMED-WITH-CAVEATS, 2 minor findings fixed pre-commit | 7d93a482... | rebased onto main (post #3086/#3094/#3103/#3110/#3119/#3123/#3130/#3139 merges), awaiting fresh CI as of 2026-09-14 |
+| 2026-09-02 | intelligence | `LearningBridge.consolidate()` reward-blind fix (hardcoded `1.0` β†’ real insight confidence via `getByKey`; round-1 fix had a wrong id/key lookup β€” `entryId` is the caller-assigned `key`, not the backend's internal `entry.id` β€” caught by adversarial critic, corrected round 2) | #3159 | #3160 | yes | ACCEPT | 56/56β†’62/62 tests (candidate); 464/465 full package (1 pre-existing environmental failure); independent critic round 2 CONFIRMED | bead6402... | rebased onto main (post #3086/#3094/#3103/#3110/#3119/#3123/#3130/#3139/#3152 merges), awaiting fresh CI as of 2026-09-14 |
diff --git a/docs/dream-cycle/dream-gist-2026-09-02.md b/docs/dream-cycle/dream-gist-2026-09-02.md
new file mode 100644
index 0000000000..8f6e01b607
--- /dev/null
+++ b/docs/dream-cycle/dream-gist-2026-09-02.md
@@ -0,0 +1,75 @@
+# Intelligence SOTA Report β€” 2026-09-02
+
+TL;DR: In 2026, the field's clearest signal on self-learning agents is **failure-derived memory** (ReasoningBank, FORGE) β€” and Ruflo's own `LearningBridge.consolidate()` is currently reward-blind, hardcoding every trajectory's completion signal to `1.0` regardless of outcome. Tonight's candidate fixes that specific defect. Separately, we confirmed a previously-evaluated fix (PR #3110, EWC gate wiring) never made it to `main` and is still an open bug in production β€” a governance finding, not a new one.
+
+## What's New in 2026
+
+| Finding | Source | Confidence |
+|---|---|---|
+| ReasoningBank distills failed trajectories into "preventative lessons," +8.3pp WebArena / +4.6pp SWE-Bench-Verified vs memory-free | [arXiv 2509.25140](https://arxiv.org/pdf/2509.25140), Google Research blog | A (whole-system effect; failure-only ablation is C) |
+| FORGE converts failures into reusable Rules/Examples artifacts without weight updates | [arXiv 2605.16233](https://arxiv.org/abs/2605.16233), May 2026 | B |
+| FOREVER: replay scheduling keyed to wall-clock/step count misaligns with real learning progress ("model time" instead) | [arXiv 2601.03938](https://arxiv.org/abs/2601.03938), Jan 2026 | B |
+| DeepSeek's auxiliary-loss-free MoE load balancing avoids routing collapse via per-expert bias | [OpenReview y1iU5czYpE](https://openreview.net/forum?id=y1iU5czYpE) | A |
+| Agent memory banks fed only successes/failures can become dominated by unsuccessful trajectories β€” a scaling hazard for any failure-ingestion design | [arXiv 2608.07169](https://arxiv.org/html/2608.07169), Aug 2026 | C |
+| Every evolutionary-self-improvement framework we checked (LangGraph, AutoGen, CrewAI, OpenAI) has EITHER never shipped human-gated promotion OR is actively retreating from it | See Competitor Comparison | A (OpenAI wind-down dates) / B (CrewAI issue thread) |
+
+## Ruflo Current Capability
+
+- `v3/@claude-flow/memory/src/learning-bridge.ts:232` (pre-fix): `await this.neural.completeTask(trajectoryId, 1.0)` β€” **every** consolidated trajectory reports success=1.0 to the neural system, whether the underlying insight was ever accessed, boosted, or decayed. There is no JUDGE step on this path.
+- Separately, `v3/@claude-flow/cli/src/memory/intelligence.ts`'s `distillLearning()` has its own, different EWC-gate defect (Fisher penalty collapses to reading one arbitrary dimension out of 384). This was found, evaluated, and ACCEPT-scoped on 2026-08-27 (issue #3109 / PR #3110) β€” but **that PR was never merged to `main`**, so the bug is still live in production. Not tonight's candidate; flagged for the human reviewer to prioritize merging #3110 specifically.
+- `persistent-sona.ts` and `learning-bridge.ts` both use honestly-labeled hash-based "embeddings" (not real semantic vectors) β€” a known, disclosed limitation, not a defect.
+- No MoE expert-load telemetry found; convergence (0.13β†’0.88 per CLAUDE.md) is measured but not consumed downstream beyond routing itself.
+
+## Competitor Comparison
+
+| Project | Self-learning / continual adaptation | Capability/permission model | Evolutionary self-improvement w/ human-gated promotion |
+|---|---|---|---|
+| LangGraph/LangChain | Partial β€” LangMem SDK, retrieval-based, no weight adaptation | Yes β€” `HumanInTheLoopMiddleware`, per-tool interrupt | None |
+| AutoGen/MS Agent Framework | None shipped (research-only) | Yes β€” `ApprovalRequiredAIFunction` | None shipped |
+| CrewAI | Partial β€” unified `Memory`, SQLite LTM | Weak β€” memory-scope only | **Declined.** Issue #3015 proposed exactly this design; closed not-planned, no rebuttal |
+| OpenAI Agents SDK/AgentKit | None (RFT is offline, not agent-level) | Strongest β€” tool-level approvals | **Retreating.** Agent Builder + Evals wind-down announced 2026-06-03, gone 2026-11-30 |
+| Qdrant | Relevance-feedback query weights (client-side, not accumulated by the store) | N/A | None |
+| Vespa | Offline-trained LTR, human-run A/B promotion | N/A | None |
+
+**Why the evolutionary-loop column is empty β€” and why that's the finding, not the gap:** DGM's own authors report their self-evolving agent hacking its reward function and fabricating logs ([arXiv 2505.22954](https://arxiv.org/abs/2505.22954)); a 2026-05 paper measures "capability erosion under self-evolution" directly ([arXiv 2605.09315](https://arxiv.org/abs/2605.09315)). CrewAI's decline and OpenAI's retreat read as a considered safety call, not an oversight. Ruflo's Darwin/Flywheel design (propose β†’ immutable receipt β†’ explicit signed promotion) is exactly the governance shape CrewAI declined and OpenAI's own cookbook admits it lacks. The honest competitive claim is against DGM/ShinkaEvolve on *containment*, not against LangGraph on raw capability β€” and the intelligence-system audit currently benchmarks SONA/MoE/HNSW speed, not capability retention across evolution generations, which is the more valuable missing benchmark here.
+
+## Hypothesis
+
+Given a `LearningBridge` consolidating active learning trajectories tied to recorded memory insights, when `consolidate()` reports each trajectory's completion reward using the insight's actual current confidence (read from backend entry metadata) instead of an unconditional constant `1.0`, then the reward passed to the neural system's `completeTask()` should track and differentiate insight confidence, relative to baseline (constant `1.0` regardless of quality), subject to: (1) `ConsolidateResult`'s public shape and counting semantics unchanged; (2) all pre-existing tests remain green; (3) missing/unreadable backend entries fall back to the prior constant `1.0`, preserving that case's behavior exactly; (4) fully deterministic, $0 test coverage.
+
+## Benchmarks
+
+Not applicable β€” this is a deterministic unit-level correctness fix, not a performance benchmark. See Evaluation.
+
+## Evaluation
+
+Real evaluator: `vitest run` against `v3/@claude-flow/memory/src/learning-bridge.test.ts`, deterministic, $0 LLM cost.
+
+- **Baseline** (pre-diff, stashed): 56/56 passing.
+- **Candidate** (post-diff, final): 62/62 passing (56 original unchanged + 6 new: real-confidence-as-reward looked up by key, differentiation across two trajectories, 3 fallback-to-1.0 cases β€” never-stored key, non-numeric confidence, throwing backend β€” and a custom-`insightNamespace` case).
+- Full package regression sweep: 464/465 passing; the 1 failure (`auto-memory-bridge.test.ts`, a read-only-file-permission test) reproduces identically on the stashed baseline β€” confirmed environmental (tests run as root in this sandbox, which bypasses the chmod the test relies on), not caused by the candidate.
+
+**Round 1 → round 2, adversarial critique caught a real bug in the fix itself.** An independent critic subagent traced the actual production call path (`AutoMemoryBridge.storeInsightInAgentDB()` returns a human-readable `key` string; `createDefaultEntry()` mints an unrelated `entry.id` UUID; `LearningBridge` receives the `key` as `entryId`) and found that round 1's `backend.get(entryId)` would almost always miss in production — `get()` is indexed by `entry.id`, not `entry.key` — silently falling back to the same `1.0` constant the fix claimed to eliminate. The unit tests passed anyway because the mock backend's `get()` was a bare stub with no id/key validation. **Fixed**: switched to `backend.getByKey(namespace, key)` (the correct index, confirmed populated at `store()` time across all three real backends — AgentDB, SQLite, and the hybrid backend that delegates to it), added a configurable `insightNamespace` (default `'learnings'`, matching the one real caller), and rewrote the mock backend's `getByKey` to do a real namespace+key lookup against stored entries so the tests exercise the actual id≠key distinction rather than trusting an unvalidated mock. Re-reviewed by the same critic: **CONFIRMED**, independently re-traced against all three backend implementations. One non-blocking nit surfaced: `controller-registry.ts` instantiates a second `LearningBridge` that doesn't thread `insightNamespace` through — but that instance is never fed any insights (dead/unwired in this registry path), so it's a pre-existing, out-of-scope gap, not something this candidate introduces.
+
+## Darwin Results
+
+Not run β€” deliberately. This is a binary correctness fix (pass the real signal, or don't) with no continuous parameter for Darwin to tune, matching last night's own precedent (PR #3152) for the same class of finding.
+
+## SOTA Proof & Witness
+
+| Field | Value |
+|---|---|
+| Session commit | `4d0134e59b4fa5e8552cb7b98c6b9846f08b0c82` |
+| Report SHA-256 (pre-witness content) | `29c9189c833c3d56c378f9a1329b3baf6f16543493ad28e4e62ec76b14c0fdb9` |
+| Witness stamp | `bead6402cb018f0e57ec6695c82e007365b5f2ae91d530a74c82a2eb8a1d362b` |
+| Evaluation receipt | `vitest run` output, this report + PR body (deterministic, reproducible via `git stash`/`git stash pop` around the diff) |
+| Flywheel evidence | No `.claude-flow/flywheel/` state in this repo; evidence retained as this diff + issue + PR + this gist |
+| Darwin lineage | Not run (binary correctness fix, no tunable parameter) |
+
+Verifier procedure: fetch the raw gist, compute SHA-256, concatenate with the session commit above, compute SHA-256 again β€” result must equal the witness stamp.
+
+## Recommended Next Steps
+
+1. **Merge PR #3110** (2026-08-27, EWC gate wiring, already evaluated ACCEPT-scoped) β€” it is still unmerged and the defect it fixes is live in production on `main`.
+2. **Investigate `LearningBridge.consolidate()`'s upstream signal quality**: now that reward reflects confidence, consider whether `onInsightAccessed`'s access-boost-only confidence model is itself a good proxy for trajectory quality, or whether a real JUDGE-style verdict (success/partial/failure, per the `intelligence.ts` `distillLearning` path) should flow into this bridge too β€” ReasoningBank-style failure ingestion (deep-research candidate #1) is the natural next step here, once #2 (tonight's fix) is confirmed merged and stable.
+3. **Fix the recovered 9-night ledger-append gap** (2026-08-24 to 2026-09-01): the Dream Cycle pipeline completed every night with a real evaluated PR, but `docs/dream-cycle/LEDGER.md` was never appended β€” identical failure mode to the 08-14..08-19 gap recovered on 08-19. Backfilled tonight; root cause of the append-step failure itself remains undiagnosed and is a standing candidate for a future `automation`/`meta` scan surface.
diff --git a/v3/@claude-flow/memory/src/learning-bridge.test.ts b/v3/@claude-flow/memory/src/learning-bridge.test.ts
index d8db9c6173..8a569cb7a9 100644
--- a/v3/@claude-flow/memory/src/learning-bridge.test.ts
+++ b/v3/@claude-flow/memory/src/learning-bridge.test.ts
@@ -51,7 +51,13 @@ function createMockBackend(): IMemoryBackend & { storedEntries: MemoryEntry[] }
storedEntries.push(entry);
}),
get: vi.fn().mockResolvedValue(null),
- getByKey: vi.fn().mockResolvedValue(null),
+ // Mirrors the real backend's keyIndex lookup (agentdb-backend.ts) so tests
+ // that store() a full entry and then look it up by its `key` (as
+ // AutoMemoryBridge does β€” see resolveConsolidationReward's id/key note)
+ // exercise the real distinction between `entry.id` and `entry.key`.
+ getByKey: vi.fn().mockImplementation(async (namespace: string, key: string) => {
+ return storedEntries.find((e) => e.namespace === namespace && e.key === key) ?? null;
+ }),
update: vi.fn().mockImplementation(async (id: string, upd: MemoryEntryUpdate) => {
const entry = storedEntries.find(e => e.id === id);
if (!entry) return null;
@@ -442,6 +448,115 @@ describe('LearningBridge', () => {
expect(result.trajectoriesCompleted).toBe(9);
});
+ function createLowThresholdBridge(threshold = 1) {
+ return new LearningBridge(backend, {
+ consolidationThreshold: threshold,
+ neuralLoader: createNeuralLoader(neural),
+ });
+ }
+
+ // These reproduce the real AutoMemoryBridge flow: the backend entry's
+ // internal `id` (a UUID) is DIFFERENT from the human-readable `key` it
+ // was stored under, and it is the `key` β€” not the `id` β€” that flows
+ // into onInsightRecorded/activeTrajectories/consolidate as "entryId".
+ // Storing via backend.store() and looking up via the real getByKey
+ // mock (keyed by namespace+key, not id) proves the fix resolves reward
+ // through the correct index rather than merely satisfying a mock that
+ // doesn't distinguish id from key.
+ it('should pass the entry\'s current confidence as the completion reward, looked up by key (not id)', async () => {
+ const lowThresholdBridge = createLowThresholdBridge();
+ const entry = createTestEntry({
+ id: 'uuid-independent-of-key',
+ key: 'insight:debugging:1:0',
+ metadata: { confidence: 0.42 },
+ });
+ await backend.store(entry);
+ neural.beginTask.mockReturnValueOnce('traj-0');
+ await lowThresholdBridge.onInsightRecorded(createTestInsight(), entry.key);
+
+ await lowThresholdBridge.consolidate();
+
+ expect(backend.getByKey).toHaveBeenCalledWith('learnings', entry.key);
+ expect(neural.completeTask).toHaveBeenCalledWith('traj-0', 0.42);
+ lowThresholdBridge.destroy();
+ });
+
+ it('should differentiate reward per trajectory based on each entry\'s confidence', async () => {
+ const lowThresholdBridge = createLowThresholdBridge(2);
+ const entryA = createTestEntry({ id: 'uuid-a', key: 'insight:a:1:0', metadata: { confidence: 0.9 } });
+ const entryB = createTestEntry({ id: 'uuid-b', key: 'insight:b:2:1', metadata: { confidence: 0.2 } });
+ await backend.store(entryA);
+ await backend.store(entryB);
+ neural.beginTask.mockReturnValueOnce('traj-0').mockReturnValueOnce('traj-1');
+ await lowThresholdBridge.onInsightRecorded(createTestInsight(), entryA.key);
+ await lowThresholdBridge.onInsightRecorded(createTestInsight({ summary: 'S2' }), entryB.key);
+
+ await lowThresholdBridge.consolidate();
+
+ expect(neural.completeTask).toHaveBeenCalledWith('traj-0', 0.9);
+ expect(neural.completeTask).toHaveBeenCalledWith('traj-1', 0.2);
+ lowThresholdBridge.destroy();
+ });
+
+ it('should fall back to reward=1.0 when no entry was ever stored under that key (unchanged prior behavior)', async () => {
+ const lowThresholdBridge = createLowThresholdBridge();
+ neural.beginTask.mockReturnValueOnce('traj-0');
+ await lowThresholdBridge.onInsightRecorded(createTestInsight(), 'insight:never-stored:1:0');
+
+ const result = await lowThresholdBridge.consolidate();
+
+ expect(neural.completeTask).toHaveBeenCalledWith('traj-0', 1.0);
+ expect(result.trajectoriesCompleted).toBe(1);
+ expect(result.patternsLearned).toBe(1);
+ lowThresholdBridge.destroy();
+ });
+
+ it('should fall back to reward=1.0 when the stored entry lacks numeric confidence', async () => {
+ const lowThresholdBridge = createLowThresholdBridge();
+ const entry = createTestEntry({ id: 'uuid-c', key: 'insight:c:1:0', metadata: {} });
+ await backend.store(entry);
+ neural.beginTask.mockReturnValueOnce('traj-0');
+ await lowThresholdBridge.onInsightRecorded(createTestInsight(), entry.key);
+
+ await lowThresholdBridge.consolidate();
+
+ expect(neural.completeTask).toHaveBeenCalledWith('traj-0', 1.0);
+ lowThresholdBridge.destroy();
+ });
+
+ it('should fall back to reward=1.0 when the backend lookup throws', async () => {
+ const lowThresholdBridge = createLowThresholdBridge();
+ neural.beginTask.mockReturnValueOnce('traj-0');
+ await lowThresholdBridge.onInsightRecorded(createTestInsight(), 'insight:d:1:0');
+ (backend.getByKey as any).mockRejectedValueOnce(new Error('DB unavailable'));
+
+ const result = await lowThresholdBridge.consolidate();
+
+ expect(neural.completeTask).toHaveBeenCalledWith('traj-0', 1.0);
+ expect(result.trajectoriesCompleted).toBe(1);
+ lowThresholdBridge.destroy();
+ });
+
+ it('should respect a custom insightNamespace when resolving reward', async () => {
+ const customNsBridge = new LearningBridge(backend, {
+ consolidationThreshold: 1,
+ insightNamespace: 'custom-ns',
+ neuralLoader: createNeuralLoader(neural),
+ });
+ const entry = createTestEntry({
+ id: 'uuid-e', key: 'insight:e:1:0', namespace: 'custom-ns', metadata: { confidence: 0.77 },
+ });
+ await backend.store(entry);
+ neural.beginTask.mockReturnValueOnce('traj-0');
+ await customNsBridge.onInsightRecorded(createTestInsight(), entry.key);
+
+ await customNsBridge.consolidate();
+
+ expect(backend.getByKey).toHaveBeenCalledWith('custom-ns', entry.key);
+ expect(neural.completeTask).toHaveBeenCalledWith('traj-0', 0.77);
+ customNsBridge.destroy();
+ });
+
it('should respect custom consolidationThreshold', async () => {
const customBridge = new LearningBridge(backend, {
consolidationThreshold: 2,
diff --git a/v3/@claude-flow/memory/src/learning-bridge.ts b/v3/@claude-flow/memory/src/learning-bridge.ts
index 6e4f32ea59..bd2eaf2121 100644
--- a/v3/@claude-flow/memory/src/learning-bridge.ts
+++ b/v3/@claude-flow/memory/src/learning-bridge.ts
@@ -39,6 +39,15 @@ export interface LearningBridgeConfig {
consolidationThreshold?: number;
/** Enable the bridge (default: true). When false all methods are no-ops */
enabled?: boolean;
+ /**
+ * Namespace insights are stored under by the caller (default: 'learnings',
+ * matching AutoMemoryBridge.storeInsightInAgentDB). Entry IDs passed into
+ * this bridge (onInsightRecorded/onInsightAccessed/consolidate) are the
+ * human-readable `key` the caller stored the entry under, not the backend's
+ * internal `entry.id` β€” so lookups must go through getByKey(namespace, key)
+ * rather than get(id).
+ */
+ insightNamespace?: string;
/**
* Optional factory for the neural learning system.
* When provided, this replaces the default dynamic import of @claude-flow/neural.
@@ -90,6 +99,7 @@ const DEFAULT_CONFIG: ResolvedConfig = {
ewcLambda: 2000,
consolidationThreshold: 10,
enabled: true,
+ insightNamespace: 'learnings',
};
const MS_PER_HOUR = 3_600_000;
@@ -229,7 +239,8 @@ export class LearningBridge extends EventEmitter {
const entries = Array.from(this.activeTrajectories.entries());
for (const [entryId, trajectoryId] of entries) {
try {
- await this.neural.completeTask(trajectoryId, 1.0);
+ const reward = await this.resolveConsolidationReward(entryId);
+ await this.neural.completeTask(trajectoryId, reward);
completed++;
patternsLearned++;
toRemove.push(entryId);
@@ -417,6 +428,33 @@ export class LearningBridge extends EventEmitter {
}
}
+ /**
+ * Resolve the reward to report to the neural system when a trajectory
+ * completes. Uses the insight's current confidence (as tracked in
+ * backend metadata, which reflects access boosts and time decay) so the
+ * completion signal reflects how the insight actually turned out rather
+ * than an unconditional constant. Falls back to 1.0 β€” the prior hardcoded
+ * value β€” when the entry is missing or lacks a numeric confidence, so
+ * that case's behavior is unchanged.
+ *
+ * `entryId` here is the caller-assigned `key` an insight was stored
+ * under (e.g. AutoMemoryBridge.storeInsightInAgentDB), not the backend's
+ * internal `entry.id` β€” the two are generated independently
+ * (types.ts:createDefaultEntry always mints its own id). Looking this up
+ * with `backend.get(entryId)` would silently miss in production and
+ * always fall through to the 1.0 default, so this must go through
+ * getByKey(namespace, key) instead.
+ */
+ private async resolveConsolidationReward(entryId: string): Promise<number> {
+ try {
+ const entry = await this.backend.getByKey(this.config.insightNamespace, entryId);
+ const confidence = entry?.metadata?.confidence;
+ return typeof confidence === 'number' ? confidence : 1.0;
+ } catch {
+ return 1.0;
+ }
+ }
+
/**
* Create a deterministic hash-based embedding for content.
* This is a lightweight stand-in for a real embedding model,