Skip to content

armgit / ruflo

Public Imported from GitHub
0

🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated

dream(intelligence): #3048 discounted Thompson sampling for model-router bandit (evaluated, ACCEPT-scoped) (#3049)

rUvb291ac4049bb
diff --git a/docs/dream-cycle/dream-gist-2026-08-17.md b/docs/dream-cycle/dream-gist-2026-08-17.md
new file mode 100644
index 0000000000..8cb4476125
--- /dev/null
+++ b/docs/dream-cycle/dream-gist-2026-08-17.md
@@ -0,0 +1,54 @@
+# Intelligence SOTA Report — 2026-08-17
+
+TL;DR: Tonight (2026-08-17, intelligence deep-dive) found that Ruflo's 3-tier model router (`model-router.ts`, ADR-026/143) accumulates its Beta-Bernoulli Thompson-sampling priors forever with zero temporal decay, while the codebase's *other* router (`q-learning-router.ts`) already solves the analogous problem via epsilon annealing — an internal-consistency gap, not a hypothetical one. Shipped an opt-in `priorDecay` config field (default disabled, zero behavior change) implementing discounted Thompson sampling. A 30-trial paired benchmark shows a real, statistically significant recovery-speed improvement in the low-complexity bucket after a simulated model-quality shift — but the effect does **not** generalize to the med-complexity bucket, a limitation an independent adversarial critic's feedback led directly to testing and disclosing.
+
+## What's New in 2026
+
+| Finding | Source | Confidence |
+|---|---|---|
+| Discounted Thompson Sampling remains an active, unsolved-by-default production problem | arXiv 2606.23933 (Jun 2026), arXiv 2305.10718 | A |
+| Confidence-Adaptive Routing (CARE) reallocates MoE expert budget via gate entropy | arXiv 2607.26052 (Jul 2026) | B |
+| Ghost Vectors: soft-deleted embeddings remain reconstructible in HNSW indexes | arXiv 2606.18497 (Jun 2026) | A (general finding); C (Ruflo-specific applicability, unmeasured) |
+| No orchestration framework (LangGraph/AutoGen/CrewAI/OpenAI Agents SDK) does learned/adaptive routing in its core loop — a deliberate, field-wide tradeoff, not an oversight | Live doc/repo review, 2026-08 | A |
+
+## Ruflo Current Capability
+
+`model-router.ts:recordOutcome()` does `bp[model].alpha += reward; bp[model].beta += 1-reward` with **zero decay anywhere in the 1490-line file** (confirmed by direct grep). Persisted forever to `.swarm/model-router-state.json`. Meanwhile `q-learning-router.ts` (a sibling, less-used router) already implements exponential epsilon annealing for its own exploration rate. Two more findings from tonight's architecture pass, not acted on tonight: the MoE gate (`moe-router.ts`) computes routing entropy every call but never uses it for expert-count control (dead signal); and `EWCConsolidator.pruneOldPatterns()` mutates its own pattern Map but never propagates deletions to `LocalReasoningBank` or the HNSW-backed store — three independent, unsynced "forgetting" mechanisms in one pipeline.
+
+## Competitor Comparison
+
+| Framework | Adaptive/learned routing | Forgetting mitigation | 2026 status |
+|---|---|---|---|
+| LangGraph | Developer-written conditionals / LLM-prompted routing only | RAG-style external memory; docs explicitly discourage weight-level fine-tuning | Active |
+| AutoGen / AG2 | LLM-prompted "manager" selection | `TeachableAgent` deprecated v0.12, removed v0.14 (Mar 2026 rewrite) | Maintenance-mode (upstream) |
+| CrewAI | Hierarchical/manager-based, not learned | Context/vector memory only | Active |
+| OpenAI Agents SDK | Developer-defined handoffs | External memory (no built-in continual learning) | Active |
+| Letta (MemGPT) | N/A | Self-editing 3-tier (Core/Recall/Archival) memory — explicit token-space alternative to weight-space continual learning | Active, sharpest comparator |
+
+None of the four major orchestration frameworks implement a real bandit/RL routing policy in their core loop — genuine adaptive routing exists only in standalone services (RouteLLM, MetaLLM) and academic work, never integrated. This is a deliberate cost/complexity tradeoff the field has made, not a gap nobody noticed — which makes Ruflo's own bandit (imperfect as tonight's finding shows) still ahead of the field's default posture. Letta's token-space memory is the sharpest available foil for EWC++'s weight/pattern-space approach, not a "competitors have nothing" claim.
+
+## Hypothesis
+
+Given the router's unbounded Beta-prior accumulation, when an exponential discount (`priorDecay < 1`) is applied to all bucket/model priors once per `recordOutcome()` call before that round's reward, then post-shift recovery speed after a simulated model-quality shift should improve relative to baseline, subject to: no material regression under a stationary (non-shifting) workload.
+
+## Benchmarks
+
+Bespoke deterministic simulation (`benchmarks/results/scripts/prior-decay-benchmark.mjs`) — no LLM calls, $0 cost, seeded PRNG (identical random stream fed to baseline and candidate per trial, isolating the decay math as the only variable). 1500 pre-shift rounds (simulating months of accumulated persisted history) + 300 post-shift rounds, n=30 paired trials, two complexity buckets. An independent adversarial-critic pass found and I fixed a real bug in the paired-t formula (population vs. sample stddev, inflating t by n/(n-1)) before finalizing these numbers.
+
+## Evaluation
+
+**evaluated: accepted, scoped.** Low bucket (haiku→sonnet shift): recovery 26.5→21.9 rounds (**-17.6%**, t=7.00), post-shift correct-routing rate +1.3pp (t=5.90), stationary invariant held (Δ=+0.02pp, t=0.70 — no regression). Med bucket (sonnet→opus shift): recovery essentially flat (Δ=-0.17 rounds, t=-0.74, not significant), stationary invariant held (Δ=-0.08pp; statistically distinguishable from zero at t=-3.01 but two orders of magnitude inside the pre-declared -1pp tolerance). **The mechanism's benefit is real but bucket-scoped** — it helps most where reward asymmetry is largest (haiku success reward=1.0 vs. opus=0.4 in `BANDIT_REWARDS`), which is exactly the case that produces the most entrenched stale posteriors. This was found only because an independent critic asked for med/high-bucket coverage; the first version of this benchmark tested only the low bucket and would have overclaimed generality. Shipped **disabled by default** (`priorDecay: 1`); this is an opt-in mechanism, not a default-behavior change.
+
+## Darwin Results
+
+**Skipped — scope mismatch**, same class of skip as 2026-08-16's security night. `@metaharness/darwin`'s discovered real interface (`evolve --bench <suite.json>`) evolves harness/prompt genomes against LLM-scored coding-task corpora; it has no analog for tuning a single continuous scalar (`priorDecay`) in a deterministic-math function. A γ-value sensitivity sweep (0.995/0.99/0.98) was run directly in the bespoke benchmark instead — 0.995 gave the cleanest signal-to-noise (t=7.00 vs. 6.76/3.71 for 0.99/0.98) and is the value used above.
+
+## SOTA Proof & Witness
+
+See the linked issue and PR for the full witness stamp (session commit, report hash, witness hash) and verifier procedure.
+
+## Recommended Next Steps
+
+1. **Merge tonight's opt-in `priorDecay` candidate** (draft PR) — zero-risk (disabled by default), reviewable single-file diff, evidence-backed for the low-complexity bucket, all findings from an independent adversarial critique addressed (paired-t formula fixed, NaN/negative-decay input validation added, med-bucket coverage added).
+2. **Follow-up candidate:** investigate why the med bucket shows no effect — likely `BANDIT_REWARDS`' smaller success-reward magnitudes for sonnet/opus change entrenchment dynamics; a bucket-scaled decay rate (stronger where reward asymmetry is largest) is the natural next hypothesis, explicitly flagged here rather than implemented tonight.
+3. **Follow-up candidate:** wire the MoE gate's already-computed routing entropy into its `topK` selection (Confidence-Adaptive Routing, arXiv 2607.26052) — currently dead signal, distinct from tonight's candidate, scored highly in tonight's 5-candidate ranking (2nd place).
diff --git a/v3/@claude-flow/cli/__tests__/router-bandit.test.ts b/v3/@claude-flow/cli/__tests__/router-bandit.test.ts
index d95a694871..68a33f15e9 100644
--- a/v3/@claude-flow/cli/__tests__/router-bandit.test.ts
+++ b/v3/@claude-flow/cli/__tests__/router-bandit.test.ts
@@ -18,7 +18,7 @@ import { describe, expect, it, beforeEach, afterEach } from 'vitest';
import { rmSync, mkdtempSync, writeFileSync, mkdirSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
-import { ModelRouter } from '../src/ruvector/model-router.js';
+import { ModelRouter, sampleBeta } from '../src/ruvector/model-router.js';
let cwdRestore: string;
let tmpDir: string;
@@ -188,3 +188,118 @@ describe('ModelRouter — Thompson sampling bandit (#1772, ADR-142 bucketed)', (
expect(meanHaiku).toBeGreaterThan(meanOpus);
}, 30_000);
});
+
+describe('ModelRouter — priorDecay (discounted Thompson sampling, arXiv 2305.10718)', () => {
+ beforeEach(setupTempCwd);
+ afterEach(cleanupTempCwd);
+
+ it('defaults priorDecay to 1 (no-op) — identical behavior to a router with no decay config', () => {
+ const withDefault = new ModelRouter();
+ const task = 'simple task';
+ for (let i = 0; i < 5; i++) withDefault.recordOutcome(task, 'haiku', 'success');
+ for (let i = 0; i < 3; i++) withDefault.recordOutcome(task, 'sonnet', 'failure');
+ const p = withDefault.getBanditPriors(bucketOf(withDefault, task));
+ // Same math as the pre-priorDecay assertions above: reward accumulates
+ // with no decay applied anywhere in the call chain.
+ expect(p.haiku.alpha).toBeCloseTo(6.0, 5); // 1 + 5*1.0
+ expect(p.haiku.beta).toBeCloseTo(1.0, 5);
+ expect(p.sonnet.beta).toBeCloseTo(4.0, 5); // 1 + 3*1.0 (failure reward=0)
+ });
+
+ it('decays ALL bucket/model priors once per recordOutcome call, before adding the new reward', () => {
+ const router = new ModelRouter({ priorDecay: 0.5 });
+ const task = 'simple task';
+ const bucket = bucketOf(router, task);
+ // Round 1: decay Beta(1,1)→Beta(1,1) (no-op at the uniform prior), then
+ // haiku success (reward 1.0) → alpha 1*0.5 + 1.0 = 1.5, beta 1*0.5 = 0.5.
+ router.recordOutcome(task, 'haiku', 'success');
+ let p = router.getBanditPriors(bucket);
+ expect(p.haiku.alpha).toBeCloseTo(1.5, 5);
+ expect(p.haiku.beta).toBeCloseTo(0.5, 5);
+ // Round 2: decay is applied to EVERY bucket/model first (this is the
+ // "every routing decision is one time-step for every arm" semantics) —
+ // so sonnet, though untouched in round 1's outcome, already decayed
+ // from Beta(1,1) to Beta(0.5,0.5) as a side effect of round 1's call.
+ // Now: haiku decays 1.5*0.5=0.75, 0.5*0.5=0.25 (no reward added, it
+ // wasn't this round's outcome). sonnet decays 0.5*0.5=0.25, 0.5*0.5=0.25,
+ // then gets this round's failure (reward 0) → alpha stays 0.25, beta
+ // becomes 0.25 + (1 - 0) = 1.25.
+ router.recordOutcome(task, 'sonnet', 'failure');
+ p = router.getBanditPriors(bucket);
+ expect(p.haiku.alpha).toBeCloseTo(0.75, 5);
+ expect(p.haiku.beta).toBeCloseTo(0.25, 5);
+ expect(p.sonnet.alpha).toBeCloseTo(0.25, 5);
+ expect(p.sonnet.beta).toBeCloseTo(1.25, 5);
+ });
+
+ it('floors decayed alpha/beta at PRIOR_DECAY_FLOOR (0.05) instead of collapsing to 0', () => {
+ const router = new ModelRouter({ priorDecay: 0.1 }); // aggressive decay
+ const task = 'simple task';
+ // One bucket/model (opus, in the 'low' bucket) never receives an outcome
+ // directly — but every recordOutcome call for ANY model in this bucket
+ // still decays it. Hammer haiku with outcomes many times; opus's
+ // untouched Beta(1,1) should decay toward the floor, never below it,
+ // and never go non-positive (which would break sampleBeta's Gamma draws).
+ for (let i = 0; i < 50; i++) router.recordOutcome(task, 'haiku', 'failure');
+ const p = router.getBanditPriors(bucketOf(router, task));
+ expect(p.opus.alpha).toBeCloseTo(0.05, 5);
+ expect(p.opus.beta).toBeCloseTo(0.05, 5);
+ expect(p.opus.alpha).toBeGreaterThan(0);
+ expect(p.opus.beta).toBeGreaterThan(0);
+ expect(sampleBeta(p.opus.alpha, p.opus.beta)).not.toBeNaN();
+ });
+
+ it('rejects an out-of-range priorDecay (NaN/negative/>1) by falling back to disabled (1)', () => {
+ // Caught by an independent adversarial-critic pass: Math.max's NaN-
+ // poisoning bypasses sampleBeta's own alpha<=0||beta<=0 fallback, and a
+ // negative decay pins every prior at PRIOR_DECAY_FLOOR on the first
+ // call — either would silently corrupt persisted router state forever.
+ for (const [i, bad] of [NaN, -1, 0, 1.5, Infinity, -Infinity].entries()) {
+ const router = new ModelRouter({
+ priorDecay: bad,
+ statePath: join(tmpDir, `.swarm/state-${i}.json`),
+ });
+ router.recordOutcome('t', 'haiku', 'success');
+ const p = router.getBanditPriors(bucketOf(router, 't'));
+ // Falls through to priorDecay=1 (disabled): plain accumulation, no decay.
+ expect(p.haiku.alpha).toBeCloseTo(2.0, 5);
+ expect(p.haiku.beta).toBeCloseTo(1.0, 5);
+ }
+ });
+
+ it('regression: candidate must not degrade routing under a stationary workload', async () => {
+ // Pre-declared invariant (STEP 3.3 hypothesis, checked in the receipt at
+ // benchmarks/results/prior-decay-receipt.json with n=30 paired trials):
+ // decay must not reduce accuracy when the correct model never changes.
+ // This is the fast in-suite version of that same check.
+ async function runStationary(priorDecay: number): Promise<number> {
+ const router = new ModelRouter({ priorDecay });
+ let seed = 0x2468ace;
+ const rng = () => {
+ seed |= 0;
+ seed = (seed + 0x6D2B79F5) | 0;
+ let t = Math.imul(seed ^ (seed >>> 15), 1 | seed);
+ t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
+ return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
+ };
+ const origRandom = Math.random;
+ Math.random = rng;
+ let correct = 0;
+ const N = 150;
+ try {
+ for (let i = 0; i < N; i++) {
+ const r = await router.route('fix a typo in the readme file');
+ const outcome = r.model === 'haiku' ? 'success' : 'failure';
+ router.recordOutcome('fix a typo in the readme file', r.model, outcome);
+ if (r.model === 'haiku') correct++;
+ }
+ } finally {
+ Math.random = origRandom;
+ }
+ return correct / N;
+ }
+ const baseline = await runStationary(1);
+ const candidate = await runStationary(0.995);
+ expect(candidate).toBeGreaterThanOrEqual(baseline - 0.05); // no material regression
+ }, 30_000);
+});
diff --git a/v3/@claude-flow/cli/benchmarks/results/prior-decay-receipt.json b/v3/@claude-flow/cli/benchmarks/results/prior-decay-receipt.json
new file mode 100644
index 0000000000..4f8a03c7a8
--- /dev/null
+++ b/v3/@claude-flow/cli/benchmarks/results/prior-decay-receipt.json
@@ -0,0 +1,59 @@
+{
+ "config": {
+ "TRIALS": 30,
+ "SHIFT_AT": 1500,
+ "POST_ROUNDS": 300,
+ "TOTAL_ROUNDS": 1800,
+ "STATIONARY_ROUNDS": 400,
+ "CANDIDATE_DECAY": 0.995,
+ "RECOVERY_WINDOW": 20,
+ "RECOVERY_THRESHOLD": 14
+ },
+ "scenarios": [
+ {
+ "bucket": "low",
+ "task": "fix a typo in the readme file",
+ "shift": "haiku -> sonnet",
+ "nonStationary": {
+ "baselineMeanRecoveryRound": 26.533333333333335,
+ "candidateMeanRecoveryRound": 21.866666666666667,
+ "recoveryRoundDeltaMean": 4.666666666666667,
+ "recoveryRoundDeltaT": 7,
+ "baselineMeanPostShiftCorrectRate": 0.9582222222222222,
+ "candidateMeanPostShiftCorrectRate": 0.9712222222222221,
+ "postShiftRateDeltaMean": 0.013000000000000012,
+ "postShiftRateDeltaT": 5.8956649538645785
+ },
+ "stationary": {
+ "baselineMeanCorrectRate": 0.9976666666666668,
+ "candidateMeanCorrectRate": 0.9978333333333335,
+ "deltaMean": 0.00016666666666666682,
+ "deltaT": 0.7010887416930878,
+ "invariantHeld": true
+ }
+ },
+ {
+ "bucket": "med",
+ "task": "refactor the payment processing module to support multiple currencies and add integration tests",
+ "shift": "sonnet -> opus",
+ "nonStationary": {
+ "baselineMeanRecoveryRound": 20.4,
+ "candidateMeanRecoveryRound": 20.566666666666666,
+ "recoveryRoundDeltaMean": -0.16666666666666666,
+ "recoveryRoundDeltaT": -0.7397092748646286,
+ "baselineMeanPostShiftCorrectRate": 0.9802222222222222,
+ "candidateMeanPostShiftCorrectRate": 0.9798888888888888,
+ "postShiftRateDeltaMean": -0.000333333333333341,
+ "postShiftRateDeltaT": -0.3324851555379874
+ },
+ "stationary": {
+ "baselineMeanCorrectRate": 0.9955,
+ "candidateMeanCorrectRate": 0.9946666666666669,
+ "deltaMean": -0.0008333333333333452,
+ "deltaT": -3.010398644698114,
+ "invariantHeld": true
+ }
+ }
+ ],
+ "generatedAt": "2026-08-17T06:29:03.189Z"
+}
\ No newline at end of file
diff --git a/v3/@claude-flow/cli/benchmarks/results/scripts/prior-decay-benchmark.mjs b/v3/@claude-flow/cli/benchmarks/results/scripts/prior-decay-benchmark.mjs
new file mode 100644
index 0000000000..c3cf54ac14
--- /dev/null
+++ b/v3/@claude-flow/cli/benchmarks/results/scripts/prior-decay-benchmark.mjs
@@ -0,0 +1,227 @@
+#!/usr/bin/env node
+// prior-decay-benchmark.mjs — discounted Thompson sampling (ModelRouterConfig
+// .priorDecay) vs undecayed baseline.
+//
+// Two scenario TYPES, run across two complexity buckets (low, med — see
+// SCENARIOS below), same paired-seed methodology as prior dream-cycle
+// nights' benchmark scripts (identical random stream fed to baseline and
+// candidate so any difference is attributable to the decay math, not RNG
+// noise):
+//
+// 1. STATIONARY — the "correct" model never changes. Pre-declared
+// invariant: candidate's cumulative reward must not regress vs baseline.
+// 2. NON-STATIONARY — the correct model changes once, mid-run (simulating
+// a real-world model-quality shift). Measures how many post-shift
+// rounds it takes the router to recover (trailing-20-window >=70% new
+// correct model).
+//
+// The 'med' bucket coverage (in addition to the original 'low' bucket) was
+// added after an independent adversarial-critic pass flagged that the first
+// version of this benchmark only ever exercised the 'low' bucket, so the
+// recovery-speed claim wasn't empirically validated for buckets with a
+// different BANDIT_REWARDS table (opus success=0.4 vs haiku=1.0, which
+// changes how fast a decayed-then-refed posterior separates).
+//
+// Usage:
+// cd v3/@claude-flow/cli
+// npx tsx benchmarks/results/scripts/prior-decay-benchmark.mjs [--trials N]
+
+import { ModelRouter } from '../../../src/ruvector/model-router.ts';
+import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+
+function argNum(flag, fallback) {
+ const i = process.argv.indexOf(flag);
+ return i >= 0 ? parseFloat(process.argv[i + 1]) : fallback;
+}
+
+const TRIALS = argNum('--trials', 30);
+const SHIFT_AT = argNum('--shift-at', 1500); // pre-shift rounds — simulates accumulated
+// history from a long-running persisted `.swarm/model-router-state.json` (months of
+// nightly routing decisions) before a real-world model-quality shift occurs.
+const POST_ROUNDS = argNum('--post-rounds', 300);
+const TOTAL_ROUNDS = SHIFT_AT + POST_ROUNDS;
+const STATIONARY_ROUNDS = argNum('--stationary-rounds', 400);
+const CANDIDATE_DECAY = argNum('--decay', 0.995);
+const RECOVERY_WINDOW = 20;
+const RECOVERY_THRESHOLD = 14; // >=70% of trailing window
+
+// Task strings pinned to a bucket by direct measurement (analyzeComplexity),
+// not by assumption — see the score probe referenced in the PR/issue.
+const SCENARIOS = [
+ {
+ bucket: 'low',
+ task: 'fix a typo in the readme file',
+ before: 'haiku',
+ after: 'sonnet',
+ },
+ {
+ bucket: 'med',
+ task: 'refactor the payment processing module to support multiple currencies and add integration tests',
+ before: 'sonnet',
+ after: 'opus',
+ },
+];
+
+function mulberry32(seed) {
+ let s = seed >>> 0;
+ return function () {
+ s = (s + 0x6d2b79f5) | 0;
+ let t = Math.imul(s ^ (s >>> 15), 1 | s);
+ t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
+ return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
+ };
+}
+
+function freshRouter(priorDecay, tmpDir, tag) {
+ return new ModelRouter({
+ statePath: join(tmpDir, `state-${tag}.json`),
+ priorDecay,
+ });
+}
+
+async function runNonStationaryTrial(scenario, priorDecay, seed, tmpDir, tag) {
+ Math.random = mulberry32(seed);
+ const router = freshRouter(priorDecay, tmpDir, tag);
+ let postShiftCorrectPicks = 0;
+ let postShiftRounds = 0;
+ let recoveryRound = null;
+ const window = [];
+ for (let i = 0; i < TOTAL_ROUNDS; i++) {
+ const correctModel = i < SHIFT_AT ? scenario.before : scenario.after;
+ const result = await router.route(scenario.task);
+ const picked = result.model;
+ const outcome = picked === correctModel ? 'success' : 'failure';
+ router.recordOutcome(scenario.task, picked, outcome);
+ if (i >= SHIFT_AT) {
+ postShiftRounds++;
+ const hit = picked === scenario.after ? 1 : 0;
+ postShiftCorrectPicks += hit;
+ window.push(hit);
+ if (window.length > RECOVERY_WINDOW) window.shift();
+ if (
+ recoveryRound === null &&
+ window.length === RECOVERY_WINDOW &&
+ window.reduce((a, b) => a + b, 0) >= RECOVERY_THRESHOLD
+ ) {
+ recoveryRound = i - SHIFT_AT + 1;
+ }
+ }
+ }
+ return {
+ postShiftCorrectRate: postShiftCorrectPicks / postShiftRounds,
+ recoveryRound: recoveryRound ?? TOTAL_ROUNDS - SHIFT_AT, // censored at max if never recovered
+ };
+}
+
+async function runStationaryTrial(scenario, priorDecay, seed, tmpDir, tag) {
+ Math.random = mulberry32(seed);
+ const router = freshRouter(priorDecay, tmpDir, tag);
+ let correctPicks = 0;
+ for (let i = 0; i < STATIONARY_ROUNDS; i++) {
+ const result = await router.route(scenario.task);
+ const picked = result.model;
+ const outcome = picked === scenario.before ? 'success' : 'failure';
+ router.recordOutcome(scenario.task, picked, outcome);
+ if (picked === scenario.before) correctPicks++;
+ }
+ return { correctRate: correctPicks / STATIONARY_ROUNDS };
+}
+
+function mean(xs) {
+ return xs.reduce((a, b) => a + b, 0) / xs.length;
+}
+// SAMPLE stddev (Bessel's correction, ÷(n-1)) — required for a paired
+// t-statistic. An earlier version of this function used population stddev
+// (÷n) with an incorrectly-placed correction factor, inflating every
+// reported t-value by exactly n/(n-1) (~3.4% at n=30). Caught by an
+// independent adversarial-critic pass; fixed here, not just in the receipt.
+function sampleStddev(xs) {
+ const m = mean(xs);
+ const n = xs.length;
+ return Math.sqrt(xs.reduce((a, x) => a + (x - m) ** 2, 0) / (n - 1));
+}
+// Paired t-statistic (candidate - baseline), one sample per seed.
+function pairedT(diffs) {
+ const n = diffs.length;
+ const m = mean(diffs);
+ const sd = sampleStddev(diffs) || 1e-12;
+ return m / (sd / Math.sqrt(n));
+}
+
+async function runScenario(scenario, tmpDir) {
+ const nsBaseline = [];
+ const nsCandidate = [];
+ const stBaseline = [];
+ const stCandidate = [];
+
+ for (let seed = 0; seed < TRIALS; seed++) {
+ const tag = `${scenario.bucket}-${seed}`;
+ nsBaseline.push(await runNonStationaryTrial(scenario, 1, 1000 + seed, tmpDir, `ns-base-${tag}`));
+ nsCandidate.push(
+ await runNonStationaryTrial(scenario, CANDIDATE_DECAY, 1000 + seed, tmpDir, `ns-cand-${tag}`)
+ );
+ stBaseline.push(await runStationaryTrial(scenario, 1, 2000 + seed, tmpDir, `st-base-${tag}`));
+ stCandidate.push(
+ await runStationaryTrial(scenario, CANDIDATE_DECAY, 2000 + seed, tmpDir, `st-cand-${tag}`)
+ );
+ }
+
+ const recoveryDiffs = nsBaseline.map((b, i) => b.recoveryRound - nsCandidate[i].recoveryRound);
+ const postShiftRateDiffs = nsCandidate.map(
+ (c, i) => c.postShiftCorrectRate - nsBaseline[i].postShiftCorrectRate
+ );
+ const stationaryDiffs = stCandidate.map((c, i) => c.correctRate - stBaseline[i].correctRate);
+
+ return {
+ bucket: scenario.bucket,
+ task: scenario.task,
+ shift: `${scenario.before} -> ${scenario.after}`,
+ nonStationary: {
+ baselineMeanRecoveryRound: mean(nsBaseline.map((r) => r.recoveryRound)),
+ candidateMeanRecoveryRound: mean(nsCandidate.map((r) => r.recoveryRound)),
+ recoveryRoundDeltaMean: mean(recoveryDiffs),
+ recoveryRoundDeltaT: pairedT(recoveryDiffs),
+ baselineMeanPostShiftCorrectRate: mean(nsBaseline.map((r) => r.postShiftCorrectRate)),
+ candidateMeanPostShiftCorrectRate: mean(nsCandidate.map((r) => r.postShiftCorrectRate)),
+ postShiftRateDeltaMean: mean(postShiftRateDiffs),
+ postShiftRateDeltaT: pairedT(postShiftRateDiffs),
+ },
+ stationary: {
+ baselineMeanCorrectRate: mean(stBaseline.map((r) => r.correctRate)),
+ candidateMeanCorrectRate: mean(stCandidate.map((r) => r.correctRate)),
+ deltaMean: mean(stationaryDiffs),
+ deltaT: pairedT(stationaryDiffs),
+ invariantHeld: mean(stationaryDiffs) >= -0.01, // candidate must not regress (>1pp) vs baseline
+ },
+ };
+}
+
+async function main() {
+ const tmpDir = mkdtempSync(join(tmpdir(), 'prior-decay-bench-'));
+ const origRandom = Math.random;
+ try {
+ const scenarioResults = [];
+ for (const scenario of SCENARIOS) {
+ scenarioResults.push(await runScenario(scenario, tmpDir));
+ }
+
+ const result = {
+ config: { TRIALS, SHIFT_AT, POST_ROUNDS, TOTAL_ROUNDS, STATIONARY_ROUNDS, CANDIDATE_DECAY, RECOVERY_WINDOW, RECOVERY_THRESHOLD },
+ scenarios: scenarioResults,
+ generatedAt: new Date().toISOString(),
+ };
+
+ console.log(JSON.stringify(result, null, 2));
+ writeFileSync(
+ join(new URL('.', import.meta.url).pathname, '..', 'prior-decay-receipt.json'),
+ JSON.stringify(result, null, 2)
+ );
+ } finally {
+ Math.random = origRandom;
+ rmSync(tmpDir, { recursive: true, force: true });
+ }
+}
+
+main();
diff --git a/v3/@claude-flow/cli/package.json b/v3/@claude-flow/cli/package.json
index a98cb8943b..6ac88b149e 100644
--- a/v3/@claude-flow/cli/package.json
+++ b/v3/@claude-flow/cli/package.json
@@ -93,6 +93,7 @@
"publish:all": "./scripts/publish.sh"
},
"devDependencies": {
+ "tsx": "^4.19.0",
"typescript": "^5.3.0",
"vitest": "^4.1.0"
},
@@ -112,10 +113,10 @@
"chalk": "^5.3.0",
"commander": "^12.0.0",
"cors": "^2.8.5",
- "fs-extra": "^11.2.0",
"express": ">=4.22.2",
"express-rate-limit": ">=8.4.1",
"fast-uri": "3.1.5",
+ "fs-extra": "^11.2.0",
"helmet": "^7.2.0",
"inquirer": "^9.2.0",
"semver": "7.7.3",
@@ -127,12 +128,12 @@
},
"optionalDependencies": {
"@agntcy/slim-bindings": "2.0.0-alpha.5",
- "@napi-rs/keyring": "1.3.0",
"@claude-flow/memory": "^3.0.0-alpha.22",
"@metaharness/darwin": "~0.9.0",
"@metaharness/flywheel": "~0.1.10",
"@metaharness/radio": "~0.1.0",
"@metaharness/turn-credit": "~0.1.0",
+ "@napi-rs/keyring": "1.3.0",
"agentdb": "^3.0.0-alpha.17",
"agentic-flow": "^3.0.0-alpha.1",
"better-sqlite3": "^12.9.0",
diff --git a/v3/@claude-flow/cli/src/ruvector/model-router.ts b/v3/@claude-flow/cli/src/ruvector/model-router.ts
index b7eb838399..1a1ec9b5c6 100644
--- a/v3/@claude-flow/cli/src/ruvector/model-router.ts
+++ b/v3/@claude-flow/cli/src/ruvector/model-router.ts
@@ -205,6 +205,18 @@ export interface ModelRouterConfig {
enableCostOptimization: boolean;
/** Prefer faster models when confidence is high (default: true) */
preferSpeed: boolean;
+ /**
+ * Discounted Thompson sampling factor applied to every bucket/model's
+ * α/β on each recordOutcome call, BEFORE that call's own reward is added
+ * (Discounted TS — arXiv 2305.10718): `α ← γα`, `β ← γβ`. Range (0, 1];
+ * 1 = disabled (default — fully backward compatible with persisted
+ * `.swarm/model-router-state.json` state). γ<1 makes old routing history
+ * decay geometrically so the bandit recovers faster after a sustained
+ * shift in a model's real-world quality, instead of unbounded accumulated
+ * history permanently dominating the posterior. Mirrors the epsilon decay
+ * `q-learning-router.ts` already applies to its own exploration rate.
+ */
+ priorDecay: number;
}
/**
@@ -509,8 +521,15 @@ const DEFAULT_CONFIG: ModelRouterConfig = {
autoSaveInterval: 1, // Save after every decision for CLI persistence
enableCostOptimization: true,
preferSpeed: true,
+ priorDecay: 1,
};
+// Beta shape parameters must stay well clear of 0 for sampleGamma/sampleBeta
+// to remain numerically stable across a persisted, long-running state file —
+// see priorDecay's doc comment. Geometric decay alone (no reward added back)
+// would otherwise shrink an unused bucket/model's α,β toward 0 forever.
+const PRIOR_DECAY_FLOOR = 0.05;
+
// Posterior mean of a Beta(α,β) prior — used by the #2250 escalation guard
// to detect when the bandit has *learned* the escalation target is worse.
function priorMean(p: { alpha: number; beta: number }): number {
@@ -554,6 +573,17 @@ export class ModelRouter {
constructor(config: Partial<ModelRouterConfig> = {}) {
this.config = { ...DEFAULT_CONFIG, ...config };
+ // priorDecay multiplies directly into persisted α/β (recordOutcome). An
+ // out-of-range value (NaN, <=0, >1 — e.g. a bad env/config parse) would
+ // silently poison `.swarm/model-router-state.json` forever: Math.max's
+ // NaN-poisoning bypasses sampleBeta's own alpha<=0||beta<=0 fallback, and
+ // a negative value pins every prior at PRIOR_DECAY_FLOOR on the very
+ // first call. Fall through to the safe default (1 = disabled) instead of
+ // throwing, matching this file's existing malformed-config idiom (see
+ // envMaxUncertainty above).
+ if (!Number.isFinite(this.config.priorDecay) || this.config.priorDecay <= 0 || this.config.priorDecay > 1) {
+ this.config.priorDecay = 1;
+ }
this.state = this.loadState();
}
@@ -1165,6 +1195,23 @@ export class ModelRouter {
// Haiku-success > Sonnet-success > Opus-success (Opus on simple tasks
// is wasteful even when correct). Failure/escalation always β++.
if (!this.state.priors) this.state.priors = defaultBucketedPriors();
+
+ // Discounted Thompson sampling (priorDecay < 1): decay ALL bucket/model
+ // priors once per recordOutcome call, before this call's own reward is
+ // added — every routing decision is one discrete "time step" for every
+ // arm, not just the one that got pulled, matching the arXiv 2305.10718
+ // formulation. No-op when priorDecay is the default of 1.
+ if (this.config.priorDecay < 1) {
+ for (const bk of Object.keys(this.state.priors) as ComplexityBucket[]) {
+ const bucketPriors = this.state.priors[bk];
+ for (const m of Object.keys(bucketPriors) as ClaudeModel[]) {
+ const p = bucketPriors[m];
+ p.alpha = Math.max(PRIOR_DECAY_FLOOR, p.alpha * this.config.priorDecay);
+ p.beta = Math.max(PRIOR_DECAY_FLOOR, p.beta * this.config.priorDecay);
+ }
+ }
+ }
+
const bp = this.state.priors[bucket] ?? (this.state.priors[bucket] = defaultBanditPriors());
const reward = BANDIT_REWARDS[model]?.[outcome] ?? 0.5;
bp[model].alpha += reward;
diff --git a/v3/pnpm-lock.yaml b/v3/pnpm-lock.yaml
index 80cb892a8f..8263162695 100644
--- a/v3/pnpm-lock.yaml
+++ b/v3/pnpm-lock.yaml
@@ -44,7 +44,7 @@ importers:
version: 5.9.3
vitest:
specifier: '>=4.1.0'
- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
'@claude-flow/aidefence':
dependencies:
@@ -110,7 +110,7 @@ importers:
version: 5.9.3
vitest:
specifier: '>=4.1.0'
- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
'@claude-flow/cli':
dependencies:
@@ -236,6 +236,9 @@ importers:
specifier: ^0.2.27
version: 0.2.27([email protected])
devDependencies:
+ tsx:
+ specifier: ^4.19.0
+ version: 4.21.0
typescript:
specifier: ^5.3.0
version: 5.9.3
@@ -286,7 +289,7 @@ importers:
version: 5.9.3
vitest:
specifier: '>=4.1.0'
- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
'@claude-flow/deployment':
dependencies:
@@ -299,7 +302,7 @@ importers:
version: 25.0.2([email protected])
vitest:
specifier: '>=4.1.0'
- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
'@claude-flow/embeddings':
dependencies:
@@ -349,7 +352,7 @@ importers:
version: 5.9.3
vitest:
specifier: '>=4.1.0'
- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
'@claude-flow/hooks':
dependencies:
@@ -384,7 +387,7 @@ importers:
version: 5.9.3
vitest:
specifier: '>=4.1.0'
- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
'@claude-flow/integration':
dependencies:
@@ -432,7 +435,7 @@ importers:
version: 8.18.1
vitest:
specifier: '>=4.1.0'
- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
'@claude-flow/memory':
dependencies:
@@ -484,7 +487,7 @@ importers:
devDependencies:
vitest:
specifier: '>=4.1.0'
- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
'@claude-flow/plugin-agent-federation':
dependencies:
@@ -534,7 +537,7 @@ importers:
version: 5.9.3
vitest:
specifier: '>=4.1.0'
- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
'@claude-flow/plugins':
dependencies:
@@ -562,7 +565,7 @@ importers:
version: 5.9.3
vitest:
specifier: '>=4.1.0'
- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
'@claude-flow/providers':
dependencies:
@@ -587,7 +590,7 @@ importers:
version: 5.9.3
vitest:
specifier: '>=4.1.0'
- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
'@claude-flow/security':
dependencies:
@@ -607,7 +610,7 @@ importers:
devDependencies:
vitest:
specifier: '>=4.1.0'
- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
'@claude-flow/shared':
dependencies:
@@ -641,7 +644,7 @@ importers:
version: 7.2.0
vitest:
specifier: '>=4.1.0'
- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
ws:
specifier: ^8.16.0
version: 8.19.0
@@ -653,7 +656,7 @@ importers:
devDependencies:
vitest:
specifier: '>=4.1.0'
- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
'@claude-flow/testing':
devDependencies:
@@ -671,7 +674,9 @@ importers:
version: 5.9.3
vitest:
specifier: '>=4.1.0'
- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+
+ '@claude-flow/watermark': {}
packages:
@@ -1775,6 +1780,7 @@ packages:
/@metaharness/[email protected]:
resolution: {integrity: sha512-yrLXDXNdf4jKqHlW6QZw3Cy5T0eMMjjaPqn1XB717/Xx/y0Iv9R03W0OhMEiR/kfqMNebRBW1fi6D7inStEyqA==}
+ requiresBuild: true
dev: false
/@metaharness/[email protected]:
@@ -6784,7 +6790,7 @@ packages:
dependencies:
'@rolldown/pluginutils': 1.0.0-beta.47
'@swc/core': 1.15.8
- vite: 7.3.0(@types/[email protected])([email protected])
transitivePeerDependencies:
- '@swc/helpers'
dev: false
@@ -6808,7 +6814,7 @@ packages:
obug: 2.1.1
std-env: 4.1.0
tinyrainbow: 3.1.0
- vitest: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
+ vitest: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])
dev: true
@@ -6836,7 +6842,7 @@ packages:
'@vitest/spy': 4.1.8
estree-walker: 3.0.3
magic-string: 0.30.21
- vite: 7.3.0(@types/[email protected])([email protected])
dev: true
@@ -7259,7 +7265,7 @@ packages:
tailwindcss-animate: 1.0.7([email protected])
tiktoken: 1.0.22
ulid: 3.0.2
- vite: 7.3.0(@types/[email protected])([email protected])
ws: 8.21.1
yaml: 2.8.2
zod: 3.25.76
@@ -9356,7 +9362,6 @@ packages:
resolution: {integrity: sha512-1VKTZJCwBrvbd+Wn3AOgQP/2Av+TfTCOlE4AcRJE72W1ksZXbAx8PPBR9RzgTeSPzlPMHrbANMH3LbltH73wxQ==}
dependencies:
resolve-pkg-maps: 1.0.0
- dev: true
resolution: {integrity: sha512-PI+sPDvHXNPl5WNOErAK05s3j0lgwUzMN6o8cyQrDaKfT3qd7TmNJKeXX+SknI5I0QhG5fVPAEwSY4tRGDtYoQ==}
@@ -11684,7 +11689,6 @@ packages:
resolution: {integrity: sha512-seS2Tj26TBVOC2NIc2rOe2y2ZO7efxITtLZcGSOnHHNOQ7CkiUBfw0Iw2ck6xkIhPwLhKNLS8BO+hEpngQlqzw==}
- dev: true
resolution: {integrity: sha512-RfqAvLnMl313r7c9oclB1HhUEAezcpLjz95wFH4LVuhk9JF/r22qmVP9AMmOU4vMX7Q8pN8jwNg/CSpdFnMjTQ==}
@@ -12778,7 +12782,6 @@ packages:
get-tsconfig: 4.13.0
optionalDependencies:
fsevents: 2.3.3
- dev: true
resolution: {integrity: sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==}
@@ -12977,7 +12980,7 @@ packages:
resolution: {integrity: sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==}
engines: {node: '>= 0.8'}
resolution: {integrity: sha512-dZwN5L1VlUBewiP6H9s2+B3e3Jg96D0vzN+Ry73sOefebhYr9f94wwkMNN/9ouoU8pV1BqA1d1zGk8928cx0rg==}
engines: {node: ^20.19.0 || >=22.12.0}
hasBin: true
@@ -13025,61 +13028,9 @@ packages:
rollup: 4.54.0
tinyglobby: 0.2.15
tsx: 4.21.0
- optionalDependencies:
- fsevents: 2.3.3
- dev: true
-
- resolution: {integrity: sha512-dZwN5L1VlUBewiP6H9s2+B3e3Jg96D0vzN+Ry73sOefebhYr9f94wwkMNN/9ouoU8pV1BqA1d1zGk8928cx0rg==}
- engines: {node: ^20.19.0 || >=22.12.0}
- hasBin: true
- peerDependencies:
- '@types/node': ^20.19.0 || >=22.12.0
- jiti: '>=1.21.0'
- less: ^4.0.0
- lightningcss: ^1.21.0
- sass: ^1.70.0
- sass-embedded: ^1.70.0
- stylus: '>=0.54.8'
- sugarss: ^5.0.0
- terser: ^5.16.0
- tsx: ^4.8.1
- yaml: ^2.4.2
- peerDependenciesMeta:
- '@types/node':
- optional: true
- jiti:
- optional: true
- less:
- optional: true
- lightningcss:
- optional: true
- sass:
- optional: true
- sass-embedded:
- optional: true
- stylus:
- optional: true
- sugarss:
- optional: true
- terser:
- optional: true
- tsx:
- optional: true
- yaml:
- optional: true
- dependencies:
- '@types/node': 20.19.27
- esbuild: 0.27.2
- fdir: 6.5.0([email protected])
- picomatch: 4.0.3
- postcss: 8.5.6
- rollup: 4.54.0
- tinyglobby: 0.2.15
yaml: 2.8.2
optionalDependencies:
fsevents: 2.3.3
- dev: false
resolution: {integrity: sha512-flY6ScbCIt9HThs+C5HS7jvGOB560DJtk/Z15IQROTA6zEy49Nh8T/dofWTQL+n3vswqn87sbJNiuqw1SDp5Ig==}
@@ -13143,7 +13094,7 @@ packages:
tinyexec: 1.0.2
tinyglobby: 0.2.15
tinyrainbow: 3.1.0
- vite: 7.3.0(@types/[email protected])([email protected])
why-is-node-running: 2.3.0
transitivePeerDependencies:
- msw
@@ -13211,7 +13162,7 @@ packages:
tinyexec: 1.0.2
tinyglobby: 0.2.15
tinyrainbow: 3.1.0
- vite: 7.3.0(@types/[email protected])([email protected])
why-is-node-running: 2.3.0
transitivePeerDependencies:
- msw