🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated
TypeScript agentic-aiagentic-frameworkagentic-workflowagentsai-agentsai-assistantai-skillsautonomous-agentsclaudecodexdsh-pluginharnessmcp-servermulti-agentmulti-agent-systemsnpmskillsswarmswarm-intelligencetypescript
dream(intelligence): #3048 discounted Thompson sampling for model-router bandit (evaluated, ACCEPT-scoped) (#3049)
rUvb291ac4049bb
diff --git a/docs/dream-cycle/dream-gist-2026-08-17.md b/docs/dream-cycle/dream-gist-2026-08-17.mdnew file mode 100644index 0000000000..8cb4476125--- /dev/null+++ b/docs/dream-cycle/dream-gist-2026-08-17.md@@ -0,0 +1,54 @@+# Intelligence SOTA Report — 2026-08-17++TL;DR: Tonight (2026-08-17, intelligence deep-dive) found that Ruflo's 3-tier model router (`model-router.ts`, ADR-026/143) accumulates its Beta-Bernoulli Thompson-sampling priors forever with zero temporal decay, while the codebase's *other* router (`q-learning-router.ts`) already solves the analogous problem via epsilon annealing — an internal-consistency gap, not a hypothetical one. Shipped an opt-in `priorDecay` config field (default disabled, zero behavior change) implementing discounted Thompson sampling. A 30-trial paired benchmark shows a real, statistically significant recovery-speed improvement in the low-complexity bucket after a simulated model-quality shift — but the effect does **not** generalize to the med-complexity bucket, a limitation an independent adversarial critic's feedback led directly to testing and disclosing.++## What's New in 2026++| Finding | Source | Confidence |+|---|---|---|+| Discounted Thompson Sampling remains an active, unsolved-by-default production problem | arXiv 2606.23933 (Jun 2026), arXiv 2305.10718 | A |+| Confidence-Adaptive Routing (CARE) reallocates MoE expert budget via gate entropy | arXiv 2607.26052 (Jul 2026) | B |+| Ghost Vectors: soft-deleted embeddings remain reconstructible in HNSW indexes | arXiv 2606.18497 (Jun 2026) | A (general finding); C (Ruflo-specific applicability, unmeasured) |+| No orchestration framework (LangGraph/AutoGen/CrewAI/OpenAI Agents SDK) does learned/adaptive routing in its core loop — a deliberate, field-wide tradeoff, not an oversight | Live doc/repo review, 2026-08 | A |++## Ruflo Current Capability++`model-router.ts:recordOutcome()` does `bp[model].alpha += reward; bp[model].beta += 1-reward` with **zero decay anywhere in the 1490-line file** (confirmed by direct grep). Persisted forever to `.swarm/model-router-state.json`. Meanwhile `q-learning-router.ts` (a sibling, less-used router) already implements exponential epsilon annealing for its own exploration rate. Two more findings from tonight's architecture pass, not acted on tonight: the MoE gate (`moe-router.ts`) computes routing entropy every call but never uses it for expert-count control (dead signal); and `EWCConsolidator.pruneOldPatterns()` mutates its own pattern Map but never propagates deletions to `LocalReasoningBank` or the HNSW-backed store — three independent, unsynced "forgetting" mechanisms in one pipeline.++## Competitor Comparison++| Framework | Adaptive/learned routing | Forgetting mitigation | 2026 status |+|---|---|---|---|+| LangGraph | Developer-written conditionals / LLM-prompted routing only | RAG-style external memory; docs explicitly discourage weight-level fine-tuning | Active |+| AutoGen / AG2 | LLM-prompted "manager" selection | `TeachableAgent` deprecated v0.12, removed v0.14 (Mar 2026 rewrite) | Maintenance-mode (upstream) |+| CrewAI | Hierarchical/manager-based, not learned | Context/vector memory only | Active |+| OpenAI Agents SDK | Developer-defined handoffs | External memory (no built-in continual learning) | Active |+| Letta (MemGPT) | N/A | Self-editing 3-tier (Core/Recall/Archival) memory — explicit token-space alternative to weight-space continual learning | Active, sharpest comparator |++None of the four major orchestration frameworks implement a real bandit/RL routing policy in their core loop — genuine adaptive routing exists only in standalone services (RouteLLM, MetaLLM) and academic work, never integrated. This is a deliberate cost/complexity tradeoff the field has made, not a gap nobody noticed — which makes Ruflo's own bandit (imperfect as tonight's finding shows) still ahead of the field's default posture. Letta's token-space memory is the sharpest available foil for EWC++'s weight/pattern-space approach, not a "competitors have nothing" claim.++## Hypothesis++Given the router's unbounded Beta-prior accumulation, when an exponential discount (`priorDecay < 1`) is applied to all bucket/model priors once per `recordOutcome()` call before that round's reward, then post-shift recovery speed after a simulated model-quality shift should improve relative to baseline, subject to: no material regression under a stationary (non-shifting) workload.++## Benchmarks++Bespoke deterministic simulation (`benchmarks/results/scripts/prior-decay-benchmark.mjs`) — no LLM calls, $0 cost, seeded PRNG (identical random stream fed to baseline and candidate per trial, isolating the decay math as the only variable). 1500 pre-shift rounds (simulating months of accumulated persisted history) + 300 post-shift rounds, n=30 paired trials, two complexity buckets. An independent adversarial-critic pass found and I fixed a real bug in the paired-t formula (population vs. sample stddev, inflating t by n/(n-1)) before finalizing these numbers.++## Evaluation++**evaluated: accepted, scoped.** Low bucket (haiku→sonnet shift): recovery 26.5→21.9 rounds (**-17.6%**, t=7.00), post-shift correct-routing rate +1.3pp (t=5.90), stationary invariant held (Δ=+0.02pp, t=0.70 — no regression). Med bucket (sonnet→opus shift): recovery essentially flat (Δ=-0.17 rounds, t=-0.74, not significant), stationary invariant held (Δ=-0.08pp; statistically distinguishable from zero at t=-3.01 but two orders of magnitude inside the pre-declared -1pp tolerance). **The mechanism's benefit is real but bucket-scoped** — it helps most where reward asymmetry is largest (haiku success reward=1.0 vs. opus=0.4 in `BANDIT_REWARDS`), which is exactly the case that produces the most entrenched stale posteriors. This was found only because an independent critic asked for med/high-bucket coverage; the first version of this benchmark tested only the low bucket and would have overclaimed generality. Shipped **disabled by default** (`priorDecay: 1`); this is an opt-in mechanism, not a default-behavior change.++## Darwin Results++**Skipped — scope mismatch**, same class of skip as 2026-08-16's security night. `@metaharness/darwin`'s discovered real interface (`evolve --bench <suite.json>`) evolves harness/prompt genomes against LLM-scored coding-task corpora; it has no analog for tuning a single continuous scalar (`priorDecay`) in a deterministic-math function. A γ-value sensitivity sweep (0.995/0.99/0.98) was run directly in the bespoke benchmark instead — 0.995 gave the cleanest signal-to-noise (t=7.00 vs. 6.76/3.71 for 0.99/0.98) and is the value used above.++## SOTA Proof & Witness++See the linked issue and PR for the full witness stamp (session commit, report hash, witness hash) and verifier procedure.++## Recommended Next Steps++1. **Merge tonight's opt-in `priorDecay` candidate** (draft PR) — zero-risk (disabled by default), reviewable single-file diff, evidence-backed for the low-complexity bucket, all findings from an independent adversarial critique addressed (paired-t formula fixed, NaN/negative-decay input validation added, med-bucket coverage added).+2. **Follow-up candidate:** investigate why the med bucket shows no effect — likely `BANDIT_REWARDS`' smaller success-reward magnitudes for sonnet/opus change entrenchment dynamics; a bucket-scaled decay rate (stronger where reward asymmetry is largest) is the natural next hypothesis, explicitly flagged here rather than implemented tonight.+3. **Follow-up candidate:** wire the MoE gate's already-computed routing entropy into its `topK` selection (Confidence-Adaptive Routing, arXiv 2607.26052) — currently dead signal, distinct from tonight's candidate, scored highly in tonight's 5-candidate ranking (2nd place).diff --git a/v3/@claude-flow/cli/__tests__/router-bandit.test.ts b/v3/@claude-flow/cli/__tests__/router-bandit.test.tsindex d95a694871..68a33f15e9 100644--- a/v3/@claude-flow/cli/__tests__/router-bandit.test.ts+++ b/v3/@claude-flow/cli/__tests__/router-bandit.test.ts@@ -18,7 +18,7 @@ import { describe, expect, it, beforeEach, afterEach } from 'vitest';import { rmSync, mkdtempSync, writeFileSync, mkdirSync } from 'node:fs';import { tmpdir } from 'node:os';import { join } from 'node:path';-import { ModelRouter } from '../src/ruvector/model-router.js';+import { ModelRouter, sampleBeta } from '../src/ruvector/model-router.js';let cwdRestore: string;let tmpDir: string;@@ -188,3 +188,118 @@ describe('ModelRouter — Thompson sampling bandit (#1772, ADR-142 bucketed)', (expect(meanHaiku).toBeGreaterThan(meanOpus);}, 30_000);});++describe('ModelRouter — priorDecay (discounted Thompson sampling, arXiv 2305.10718)', () => {+ beforeEach(setupTempCwd);+ afterEach(cleanupTempCwd);++ it('defaults priorDecay to 1 (no-op) — identical behavior to a router with no decay config', () => {+ const withDefault = new ModelRouter();+ const task = 'simple task';+ for (let i = 0; i < 5; i++) withDefault.recordOutcome(task, 'haiku', 'success');+ for (let i = 0; i < 3; i++) withDefault.recordOutcome(task, 'sonnet', 'failure');+ const p = withDefault.getBanditPriors(bucketOf(withDefault, task));+ // Same math as the pre-priorDecay assertions above: reward accumulates+ // with no decay applied anywhere in the call chain.+ expect(p.haiku.alpha).toBeCloseTo(6.0, 5); // 1 + 5*1.0+ expect(p.haiku.beta).toBeCloseTo(1.0, 5);+ expect(p.sonnet.beta).toBeCloseTo(4.0, 5); // 1 + 3*1.0 (failure reward=0)+ });++ it('decays ALL bucket/model priors once per recordOutcome call, before adding the new reward', () => {+ const router = new ModelRouter({ priorDecay: 0.5 });+ const task = 'simple task';+ const bucket = bucketOf(router, task);+ // Round 1: decay Beta(1,1)→Beta(1,1) (no-op at the uniform prior), then+ // haiku success (reward 1.0) → alpha 1*0.5 + 1.0 = 1.5, beta 1*0.5 = 0.5.+ router.recordOutcome(task, 'haiku', 'success');+ let p = router.getBanditPriors(bucket);+ expect(p.haiku.alpha).toBeCloseTo(1.5, 5);+ expect(p.haiku.beta).toBeCloseTo(0.5, 5);+ // Round 2: decay is applied to EVERY bucket/model first (this is the+ // "every routing decision is one time-step for every arm" semantics) —+ // so sonnet, though untouched in round 1's outcome, already decayed+ // from Beta(1,1) to Beta(0.5,0.5) as a side effect of round 1's call.+ // Now: haiku decays 1.5*0.5=0.75, 0.5*0.5=0.25 (no reward added, it+ // wasn't this round's outcome). sonnet decays 0.5*0.5=0.25, 0.5*0.5=0.25,+ // then gets this round's failure (reward 0) → alpha stays 0.25, beta+ // becomes 0.25 + (1 - 0) = 1.25.+ router.recordOutcome(task, 'sonnet', 'failure');+ p = router.getBanditPriors(bucket);+ expect(p.haiku.alpha).toBeCloseTo(0.75, 5);+ expect(p.haiku.beta).toBeCloseTo(0.25, 5);+ expect(p.sonnet.alpha).toBeCloseTo(0.25, 5);+ expect(p.sonnet.beta).toBeCloseTo(1.25, 5);+ });++ it('floors decayed alpha/beta at PRIOR_DECAY_FLOOR (0.05) instead of collapsing to 0', () => {+ const router = new ModelRouter({ priorDecay: 0.1 }); // aggressive decay+ const task = 'simple task';+ // One bucket/model (opus, in the 'low' bucket) never receives an outcome+ // directly — but every recordOutcome call for ANY model in this bucket+ // still decays it. Hammer haiku with outcomes many times; opus's+ // untouched Beta(1,1) should decay toward the floor, never below it,+ // and never go non-positive (which would break sampleBeta's Gamma draws).+ for (let i = 0; i < 50; i++) router.recordOutcome(task, 'haiku', 'failure');+ const p = router.getBanditPriors(bucketOf(router, task));+ expect(p.opus.alpha).toBeCloseTo(0.05, 5);+ expect(p.opus.beta).toBeCloseTo(0.05, 5);+ expect(p.opus.alpha).toBeGreaterThan(0);+ expect(p.opus.beta).toBeGreaterThan(0);+ expect(sampleBeta(p.opus.alpha, p.opus.beta)).not.toBeNaN();+ });++ it('rejects an out-of-range priorDecay (NaN/negative/>1) by falling back to disabled (1)', () => {+ // Caught by an independent adversarial-critic pass: Math.max's NaN-+ // poisoning bypasses sampleBeta's own alpha<=0||beta<=0 fallback, and a+ // negative decay pins every prior at PRIOR_DECAY_FLOOR on the first+ // call — either would silently corrupt persisted router state forever.+ for (const [i, bad] of [NaN, -1, 0, 1.5, Infinity, -Infinity].entries()) {+ const router = new ModelRouter({+ priorDecay: bad,+ statePath: join(tmpDir, `.swarm/state-${i}.json`),+ });+ router.recordOutcome('t', 'haiku', 'success');+ const p = router.getBanditPriors(bucketOf(router, 't'));+ // Falls through to priorDecay=1 (disabled): plain accumulation, no decay.+ expect(p.haiku.alpha).toBeCloseTo(2.0, 5);+ expect(p.haiku.beta).toBeCloseTo(1.0, 5);+ }+ });++ it('regression: candidate must not degrade routing under a stationary workload', async () => {+ // Pre-declared invariant (STEP 3.3 hypothesis, checked in the receipt at+ // benchmarks/results/prior-decay-receipt.json with n=30 paired trials):+ // decay must not reduce accuracy when the correct model never changes.+ // This is the fast in-suite version of that same check.+ async function runStationary(priorDecay: number): Promise<number> {+ const router = new ModelRouter({ priorDecay });+ let seed = 0x2468ace;+ const rng = () => {+ seed |= 0;+ seed = (seed + 0x6D2B79F5) | 0;+ let t = Math.imul(seed ^ (seed >>> 15), 1 | seed);+ t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;+ return ((t ^ (t >>> 14)) >>> 0) / 4294967296;+ };+ const origRandom = Math.random;+ Math.random = rng;+ let correct = 0;+ const N = 150;+ try {+ for (let i = 0; i < N; i++) {+ const r = await router.route('fix a typo in the readme file');+ const outcome = r.model === 'haiku' ? 'success' : 'failure';+ router.recordOutcome('fix a typo in the readme file', r.model, outcome);+ if (r.model === 'haiku') correct++;+ }+ } finally {+ Math.random = origRandom;+ }+ return correct / N;+ }+ const baseline = await runStationary(1);+ const candidate = await runStationary(0.995);+ expect(candidate).toBeGreaterThanOrEqual(baseline - 0.05); // no material regression+ }, 30_000);+});diff --git a/v3/@claude-flow/cli/benchmarks/results/prior-decay-receipt.json b/v3/@claude-flow/cli/benchmarks/results/prior-decay-receipt.jsonnew file mode 100644index 0000000000..4f8a03c7a8--- /dev/null+++ b/v3/@claude-flow/cli/benchmarks/results/prior-decay-receipt.json@@ -0,0 +1,59 @@+{+ "config": {+ "TRIALS": 30,+ "SHIFT_AT": 1500,+ "POST_ROUNDS": 300,+ "TOTAL_ROUNDS": 1800,+ "STATIONARY_ROUNDS": 400,+ "CANDIDATE_DECAY": 0.995,+ "RECOVERY_WINDOW": 20,+ "RECOVERY_THRESHOLD": 14+ },+ "scenarios": [+ {+ "bucket": "low",+ "task": "fix a typo in the readme file",+ "shift": "haiku -> sonnet",+ "nonStationary": {+ "baselineMeanRecoveryRound": 26.533333333333335,+ "candidateMeanRecoveryRound": 21.866666666666667,+ "recoveryRoundDeltaMean": 4.666666666666667,+ "recoveryRoundDeltaT": 7,+ "baselineMeanPostShiftCorrectRate": 0.9582222222222222,+ "candidateMeanPostShiftCorrectRate": 0.9712222222222221,+ "postShiftRateDeltaMean": 0.013000000000000012,+ "postShiftRateDeltaT": 5.8956649538645785+ },+ "stationary": {+ "baselineMeanCorrectRate": 0.9976666666666668,+ "candidateMeanCorrectRate": 0.9978333333333335,+ "deltaMean": 0.00016666666666666682,+ "deltaT": 0.7010887416930878,+ "invariantHeld": true+ }+ },+ {+ "bucket": "med",+ "task": "refactor the payment processing module to support multiple currencies and add integration tests",+ "shift": "sonnet -> opus",+ "nonStationary": {+ "baselineMeanRecoveryRound": 20.4,+ "candidateMeanRecoveryRound": 20.566666666666666,+ "recoveryRoundDeltaMean": -0.16666666666666666,+ "recoveryRoundDeltaT": -0.7397092748646286,+ "baselineMeanPostShiftCorrectRate": 0.9802222222222222,+ "candidateMeanPostShiftCorrectRate": 0.9798888888888888,+ "postShiftRateDeltaMean": -0.000333333333333341,+ "postShiftRateDeltaT": -0.3324851555379874+ },+ "stationary": {+ "baselineMeanCorrectRate": 0.9955,+ "candidateMeanCorrectRate": 0.9946666666666669,+ "deltaMean": -0.0008333333333333452,+ "deltaT": -3.010398644698114,+ "invariantHeld": true+ }+ }+ ],+ "generatedAt": "2026-08-17T06:29:03.189Z"+}\ No newline at end of filediff --git a/v3/@claude-flow/cli/benchmarks/results/scripts/prior-decay-benchmark.mjs b/v3/@claude-flow/cli/benchmarks/results/scripts/prior-decay-benchmark.mjsnew file mode 100644index 0000000000..c3cf54ac14--- /dev/null+++ b/v3/@claude-flow/cli/benchmarks/results/scripts/prior-decay-benchmark.mjs@@ -0,0 +1,227 @@+#!/usr/bin/env node+// prior-decay-benchmark.mjs — discounted Thompson sampling (ModelRouterConfig+// .priorDecay) vs undecayed baseline.+//+// Two scenario TYPES, run across two complexity buckets (low, med — see+// SCENARIOS below), same paired-seed methodology as prior dream-cycle+// nights' benchmark scripts (identical random stream fed to baseline and+// candidate so any difference is attributable to the decay math, not RNG+// noise):+//+// 1. STATIONARY — the "correct" model never changes. Pre-declared+// invariant: candidate's cumulative reward must not regress vs baseline.+// 2. NON-STATIONARY — the correct model changes once, mid-run (simulating+// a real-world model-quality shift). Measures how many post-shift+// rounds it takes the router to recover (trailing-20-window >=70% new+// correct model).+//+// The 'med' bucket coverage (in addition to the original 'low' bucket) was+// added after an independent adversarial-critic pass flagged that the first+// version of this benchmark only ever exercised the 'low' bucket, so the+// recovery-speed claim wasn't empirically validated for buckets with a+// different BANDIT_REWARDS table (opus success=0.4 vs haiku=1.0, which+// changes how fast a decayed-then-refed posterior separates).+//+// Usage:+// cd v3/@claude-flow/cli+// npx tsx benchmarks/results/scripts/prior-decay-benchmark.mjs [--trials N]++import { ModelRouter } from '../../../src/ruvector/model-router.ts';+import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';+import { tmpdir } from 'node:os';+import { join } from 'node:path';++function argNum(flag, fallback) {+ const i = process.argv.indexOf(flag);+ return i >= 0 ? parseFloat(process.argv[i + 1]) : fallback;+}++const TRIALS = argNum('--trials', 30);+const SHIFT_AT = argNum('--shift-at', 1500); // pre-shift rounds — simulates accumulated+// history from a long-running persisted `.swarm/model-router-state.json` (months of+// nightly routing decisions) before a real-world model-quality shift occurs.+const POST_ROUNDS = argNum('--post-rounds', 300);+const TOTAL_ROUNDS = SHIFT_AT + POST_ROUNDS;+const STATIONARY_ROUNDS = argNum('--stationary-rounds', 400);+const CANDIDATE_DECAY = argNum('--decay', 0.995);+const RECOVERY_WINDOW = 20;+const RECOVERY_THRESHOLD = 14; // >=70% of trailing window++// Task strings pinned to a bucket by direct measurement (analyzeComplexity),+// not by assumption — see the score probe referenced in the PR/issue.+const SCENARIOS = [+ {+ bucket: 'low',+ task: 'fix a typo in the readme file',+ before: 'haiku',+ after: 'sonnet',+ },+ {+ bucket: 'med',+ task: 'refactor the payment processing module to support multiple currencies and add integration tests',+ before: 'sonnet',+ after: 'opus',+ },+];++function mulberry32(seed) {+ let s = seed >>> 0;+ return function () {+ s = (s + 0x6d2b79f5) | 0;+ let t = Math.imul(s ^ (s >>> 15), 1 | s);+ t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;+ return ((t ^ (t >>> 14)) >>> 0) / 4294967296;+ };+}++function freshRouter(priorDecay, tmpDir, tag) {+ return new ModelRouter({+ statePath: join(tmpDir, `state-${tag}.json`),+ priorDecay,+ });+}++async function runNonStationaryTrial(scenario, priorDecay, seed, tmpDir, tag) {+ Math.random = mulberry32(seed);+ const router = freshRouter(priorDecay, tmpDir, tag);+ let postShiftCorrectPicks = 0;+ let postShiftRounds = 0;+ let recoveryRound = null;+ const window = [];+ for (let i = 0; i < TOTAL_ROUNDS; i++) {+ const correctModel = i < SHIFT_AT ? scenario.before : scenario.after;+ const result = await router.route(scenario.task);+ const picked = result.model;+ const outcome = picked === correctModel ? 'success' : 'failure';+ router.recordOutcome(scenario.task, picked, outcome);+ if (i >= SHIFT_AT) {+ postShiftRounds++;+ const hit = picked === scenario.after ? 1 : 0;+ postShiftCorrectPicks += hit;+ window.push(hit);+ if (window.length > RECOVERY_WINDOW) window.shift();+ if (+ recoveryRound === null &&+ window.length === RECOVERY_WINDOW &&+ window.reduce((a, b) => a + b, 0) >= RECOVERY_THRESHOLD+ ) {+ recoveryRound = i - SHIFT_AT + 1;+ }+ }+ }+ return {+ postShiftCorrectRate: postShiftCorrectPicks / postShiftRounds,+ recoveryRound: recoveryRound ?? TOTAL_ROUNDS - SHIFT_AT, // censored at max if never recovered+ };+}++async function runStationaryTrial(scenario, priorDecay, seed, tmpDir, tag) {+ Math.random = mulberry32(seed);+ const router = freshRouter(priorDecay, tmpDir, tag);+ let correctPicks = 0;+ for (let i = 0; i < STATIONARY_ROUNDS; i++) {+ const result = await router.route(scenario.task);+ const picked = result.model;+ const outcome = picked === scenario.before ? 'success' : 'failure';+ router.recordOutcome(scenario.task, picked, outcome);+ if (picked === scenario.before) correctPicks++;+ }+ return { correctRate: correctPicks / STATIONARY_ROUNDS };+}++function mean(xs) {+ return xs.reduce((a, b) => a + b, 0) / xs.length;+}+// SAMPLE stddev (Bessel's correction, ÷(n-1)) — required for a paired+// t-statistic. An earlier version of this function used population stddev+// (÷n) with an incorrectly-placed correction factor, inflating every+// reported t-value by exactly n/(n-1) (~3.4% at n=30). Caught by an+// independent adversarial-critic pass; fixed here, not just in the receipt.+function sampleStddev(xs) {+ const m = mean(xs);+ const n = xs.length;+ return Math.sqrt(xs.reduce((a, x) => a + (x - m) ** 2, 0) / (n - 1));+}+// Paired t-statistic (candidate - baseline), one sample per seed.+function pairedT(diffs) {+ const n = diffs.length;+ const m = mean(diffs);+ const sd = sampleStddev(diffs) || 1e-12;+ return m / (sd / Math.sqrt(n));+}++async function runScenario(scenario, tmpDir) {+ const nsBaseline = [];+ const nsCandidate = [];+ const stBaseline = [];+ const stCandidate = [];++ for (let seed = 0; seed < TRIALS; seed++) {+ const tag = `${scenario.bucket}-${seed}`;+ nsBaseline.push(await runNonStationaryTrial(scenario, 1, 1000 + seed, tmpDir, `ns-base-${tag}`));+ nsCandidate.push(+ await runNonStationaryTrial(scenario, CANDIDATE_DECAY, 1000 + seed, tmpDir, `ns-cand-${tag}`)+ );+ stBaseline.push(await runStationaryTrial(scenario, 1, 2000 + seed, tmpDir, `st-base-${tag}`));+ stCandidate.push(+ await runStationaryTrial(scenario, CANDIDATE_DECAY, 2000 + seed, tmpDir, `st-cand-${tag}`)+ );+ }++ const recoveryDiffs = nsBaseline.map((b, i) => b.recoveryRound - nsCandidate[i].recoveryRound);+ const postShiftRateDiffs = nsCandidate.map(+ (c, i) => c.postShiftCorrectRate - nsBaseline[i].postShiftCorrectRate+ );+ const stationaryDiffs = stCandidate.map((c, i) => c.correctRate - stBaseline[i].correctRate);++ return {+ bucket: scenario.bucket,+ task: scenario.task,+ shift: `${scenario.before} -> ${scenario.after}`,+ nonStationary: {+ baselineMeanRecoveryRound: mean(nsBaseline.map((r) => r.recoveryRound)),+ candidateMeanRecoveryRound: mean(nsCandidate.map((r) => r.recoveryRound)),+ recoveryRoundDeltaMean: mean(recoveryDiffs),+ recoveryRoundDeltaT: pairedT(recoveryDiffs),+ baselineMeanPostShiftCorrectRate: mean(nsBaseline.map((r) => r.postShiftCorrectRate)),+ candidateMeanPostShiftCorrectRate: mean(nsCandidate.map((r) => r.postShiftCorrectRate)),+ postShiftRateDeltaMean: mean(postShiftRateDiffs),+ postShiftRateDeltaT: pairedT(postShiftRateDiffs),+ },+ stationary: {+ baselineMeanCorrectRate: mean(stBaseline.map((r) => r.correctRate)),+ candidateMeanCorrectRate: mean(stCandidate.map((r) => r.correctRate)),+ deltaMean: mean(stationaryDiffs),+ deltaT: pairedT(stationaryDiffs),+ invariantHeld: mean(stationaryDiffs) >= -0.01, // candidate must not regress (>1pp) vs baseline+ },+ };+}++async function main() {+ const tmpDir = mkdtempSync(join(tmpdir(), 'prior-decay-bench-'));+ const origRandom = Math.random;+ try {+ const scenarioResults = [];+ for (const scenario of SCENARIOS) {+ scenarioResults.push(await runScenario(scenario, tmpDir));+ }++ const result = {+ config: { TRIALS, SHIFT_AT, POST_ROUNDS, TOTAL_ROUNDS, STATIONARY_ROUNDS, CANDIDATE_DECAY, RECOVERY_WINDOW, RECOVERY_THRESHOLD },+ scenarios: scenarioResults,+ generatedAt: new Date().toISOString(),+ };++ console.log(JSON.stringify(result, null, 2));+ writeFileSync(+ join(new URL('.', import.meta.url).pathname, '..', 'prior-decay-receipt.json'),+ JSON.stringify(result, null, 2)+ );+ } finally {+ Math.random = origRandom;+ rmSync(tmpDir, { recursive: true, force: true });+ }+}++main();diff --git a/v3/@claude-flow/cli/package.json b/v3/@claude-flow/cli/package.jsonindex a98cb8943b..6ac88b149e 100644--- a/v3/@claude-flow/cli/package.json+++ b/v3/@claude-flow/cli/package.json@@ -93,6 +93,7 @@"publish:all": "./scripts/publish.sh"},"devDependencies": {+ "tsx": "^4.19.0","typescript": "^5.3.0","vitest": "^4.1.0"},@@ -112,10 +113,10 @@"chalk": "^5.3.0","commander": "^12.0.0","cors": "^2.8.5",- "fs-extra": "^11.2.0","express": ">=4.22.2","express-rate-limit": ">=8.4.1","fast-uri": "3.1.5",+ "fs-extra": "^11.2.0","helmet": "^7.2.0","inquirer": "^9.2.0","semver": "7.7.3",@@ -127,12 +128,12 @@},"optionalDependencies": {"@agntcy/slim-bindings": "2.0.0-alpha.5",- "@napi-rs/keyring": "1.3.0","@claude-flow/memory": "^3.0.0-alpha.22","@metaharness/darwin": "~0.9.0","@metaharness/flywheel": "~0.1.10","@metaharness/radio": "~0.1.0","@metaharness/turn-credit": "~0.1.0",+ "@napi-rs/keyring": "1.3.0","agentdb": "^3.0.0-alpha.17","agentic-flow": "^3.0.0-alpha.1","better-sqlite3": "^12.9.0",diff --git a/v3/@claude-flow/cli/src/ruvector/model-router.ts b/v3/@claude-flow/cli/src/ruvector/model-router.tsindex b7eb838399..1a1ec9b5c6 100644--- a/v3/@claude-flow/cli/src/ruvector/model-router.ts+++ b/v3/@claude-flow/cli/src/ruvector/model-router.ts@@ -205,6 +205,18 @@ export interface ModelRouterConfig {enableCostOptimization: boolean;/** Prefer faster models when confidence is high (default: true) */preferSpeed: boolean;+ /**+ * Discounted Thompson sampling factor applied to every bucket/model's+ * α/β on each recordOutcome call, BEFORE that call's own reward is added+ * (Discounted TS — arXiv 2305.10718): `α ← γα`, `β ← γβ`. Range (0, 1];+ * 1 = disabled (default — fully backward compatible with persisted+ * `.swarm/model-router-state.json` state). γ<1 makes old routing history+ * decay geometrically so the bandit recovers faster after a sustained+ * shift in a model's real-world quality, instead of unbounded accumulated+ * history permanently dominating the posterior. Mirrors the epsilon decay+ * `q-learning-router.ts` already applies to its own exploration rate.+ */+ priorDecay: number;}/**@@ -509,8 +521,15 @@ const DEFAULT_CONFIG: ModelRouterConfig = {autoSaveInterval: 1, // Save after every decision for CLI persistenceenableCostOptimization: true,preferSpeed: true,+ priorDecay: 1,};+// Beta shape parameters must stay well clear of 0 for sampleGamma/sampleBeta+// to remain numerically stable across a persisted, long-running state file —+// see priorDecay's doc comment. Geometric decay alone (no reward added back)+// would otherwise shrink an unused bucket/model's α,β toward 0 forever.+const PRIOR_DECAY_FLOOR = 0.05;+// Posterior mean of a Beta(α,β) prior — used by the #2250 escalation guard// to detect when the bandit has *learned* the escalation target is worse.function priorMean(p: { alpha: number; beta: number }): number {@@ -554,6 +573,17 @@ export class ModelRouter {constructor(config: Partial<ModelRouterConfig> = {}) {this.config = { ...DEFAULT_CONFIG, ...config };+ // priorDecay multiplies directly into persisted α/β (recordOutcome). An+ // out-of-range value (NaN, <=0, >1 — e.g. a bad env/config parse) would+ // silently poison `.swarm/model-router-state.json` forever: Math.max's+ // NaN-poisoning bypasses sampleBeta's own alpha<=0||beta<=0 fallback, and+ // a negative value pins every prior at PRIOR_DECAY_FLOOR on the very+ // first call. Fall through to the safe default (1 = disabled) instead of+ // throwing, matching this file's existing malformed-config idiom (see+ // envMaxUncertainty above).+ if (!Number.isFinite(this.config.priorDecay) || this.config.priorDecay <= 0 || this.config.priorDecay > 1) {+ this.config.priorDecay = 1;+ }this.state = this.loadState();}@@ -1165,6 +1195,23 @@ export class ModelRouter {// Haiku-success > Sonnet-success > Opus-success (Opus on simple tasks// is wasteful even when correct). Failure/escalation always β++.if (!this.state.priors) this.state.priors = defaultBucketedPriors();++ // Discounted Thompson sampling (priorDecay < 1): decay ALL bucket/model+ // priors once per recordOutcome call, before this call's own reward is+ // added — every routing decision is one discrete "time step" for every+ // arm, not just the one that got pulled, matching the arXiv 2305.10718+ // formulation. No-op when priorDecay is the default of 1.+ if (this.config.priorDecay < 1) {+ for (const bk of Object.keys(this.state.priors) as ComplexityBucket[]) {+ const bucketPriors = this.state.priors[bk];+ for (const m of Object.keys(bucketPriors) as ClaudeModel[]) {+ const p = bucketPriors[m];+ p.alpha = Math.max(PRIOR_DECAY_FLOOR, p.alpha * this.config.priorDecay);+ p.beta = Math.max(PRIOR_DECAY_FLOOR, p.beta * this.config.priorDecay);+ }+ }+ }+const bp = this.state.priors[bucket] ?? (this.state.priors[bucket] = defaultBanditPriors());const reward = BANDIT_REWARDS[model]?.[outcome] ?? 0.5;bp[model].alpha += reward;diff --git a/v3/pnpm-lock.yaml b/v3/pnpm-lock.yamlindex 80cb892a8f..8263162695 100644--- a/v3/pnpm-lock.yaml+++ b/v3/pnpm-lock.yaml@@ -44,7 +44,7 @@ importers:version: 5.9.3vitest:specifier: '>=4.1.0'- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])'@claude-flow/aidefence':dependencies:@@ -110,7 +110,7 @@ importers:version: 5.9.3vitest:specifier: '>=4.1.0'- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])'@claude-flow/cli':dependencies:@@ -236,6 +236,9 @@ importers:specifier: ^0.2.27version: 0.2.27([email protected])devDependencies:+ tsx:+ specifier: ^4.19.0+ version: 4.21.0typescript:specifier: ^5.3.0version: 5.9.3@@ -286,7 +289,7 @@ importers:version: 5.9.3vitest:specifier: '>=4.1.0'- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])'@claude-flow/deployment':dependencies:@@ -299,7 +302,7 @@ importers:version: 25.0.2([email protected])vitest:specifier: '>=4.1.0'- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])'@claude-flow/embeddings':dependencies:@@ -349,7 +352,7 @@ importers:version: 5.9.3vitest:specifier: '>=4.1.0'- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])'@claude-flow/hooks':dependencies:@@ -384,7 +387,7 @@ importers:version: 5.9.3vitest:specifier: '>=4.1.0'- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])'@claude-flow/integration':dependencies:@@ -432,7 +435,7 @@ importers:version: 8.18.1vitest:specifier: '>=4.1.0'- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])'@claude-flow/memory':dependencies:@@ -484,7 +487,7 @@ importers:devDependencies:vitest:specifier: '>=4.1.0'- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])'@claude-flow/plugin-agent-federation':dependencies:@@ -534,7 +537,7 @@ importers:version: 5.9.3vitest:specifier: '>=4.1.0'- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])'@claude-flow/plugins':dependencies:@@ -562,7 +565,7 @@ importers:version: 5.9.3vitest:specifier: '>=4.1.0'- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])'@claude-flow/providers':dependencies:@@ -587,7 +590,7 @@ importers:version: 5.9.3vitest:specifier: '>=4.1.0'- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])'@claude-flow/security':dependencies:@@ -607,7 +610,7 @@ importers:devDependencies:vitest:specifier: '>=4.1.0'- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])'@claude-flow/shared':dependencies:@@ -641,7 +644,7 @@ importers:version: 7.2.0vitest:specifier: '>=4.1.0'- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])ws:specifier: ^8.16.0version: 8.19.0@@ -653,7 +656,7 @@ importers:devDependencies:vitest:specifier: '>=4.1.0'- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])'@claude-flow/testing':devDependencies:@@ -671,7 +674,9 @@ importers:version: 5.9.3vitest:specifier: '>=4.1.0'- version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])+ version: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])++ '@claude-flow/watermark': {}packages:@@ -1775,6 +1780,7 @@ packages:/@metaharness/[email protected]:resolution: {integrity: sha512-yrLXDXNdf4jKqHlW6QZw3Cy5T0eMMjjaPqn1XB717/Xx/y0Iv9R03W0OhMEiR/kfqMNebRBW1fi6D7inStEyqA==}+ requiresBuild: truedev: false/@metaharness/[email protected]:@@ -6784,7 +6790,7 @@ packages:dependencies:'@rolldown/pluginutils': 1.0.0-beta.47'@swc/core': 1.15.8- vite: 7.3.0(@types/[email protected])([email protected])transitivePeerDependencies:- '@swc/helpers'dev: false@@ -6808,7 +6814,7 @@ packages:obug: 2.1.1std-env: 4.1.0tinyrainbow: 3.1.0- vitest: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])+ vitest: 4.1.8(@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected])dev: true/@vitest/[email protected]:@@ -6836,7 +6842,7 @@ packages:'@vitest/spy': 4.1.8estree-walker: 3.0.3magic-string: 0.30.21- vite: 7.3.0(@types/[email protected])([email protected])dev: true/@vitest/[email protected]:@@ -7259,7 +7265,7 @@ packages:tailwindcss-animate: 1.0.7([email protected])tiktoken: 1.0.22ulid: 3.0.2- vite: 7.3.0(@types/[email protected])([email protected])ws: 8.21.1yaml: 2.8.2zod: 3.25.76@@ -9356,7 +9362,6 @@ packages:resolution: {integrity: sha512-1VKTZJCwBrvbd+Wn3AOgQP/2Av+TfTCOlE4AcRJE72W1ksZXbAx8PPBR9RzgTeSPzlPMHrbANMH3LbltH73wxQ==}dependencies:resolve-pkg-maps: 1.0.0- dev: trueresolution: {integrity: sha512-PI+sPDvHXNPl5WNOErAK05s3j0lgwUzMN6o8cyQrDaKfT3qd7TmNJKeXX+SknI5I0QhG5fVPAEwSY4tRGDtYoQ==}@@ -11684,7 +11689,6 @@ packages:resolution: {integrity: sha512-seS2Tj26TBVOC2NIc2rOe2y2ZO7efxITtLZcGSOnHHNOQ7CkiUBfw0Iw2ck6xkIhPwLhKNLS8BO+hEpngQlqzw==}- dev: trueresolution: {integrity: sha512-RfqAvLnMl313r7c9oclB1HhUEAezcpLjz95wFH4LVuhk9JF/r22qmVP9AMmOU4vMX7Q8pN8jwNg/CSpdFnMjTQ==}@@ -12778,7 +12782,6 @@ packages:get-tsconfig: 4.13.0optionalDependencies:fsevents: 2.3.3- dev: trueresolution: {integrity: sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w==}@@ -12977,7 +12980,7 @@ packages:resolution: {integrity: sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==}engines: {node: '>= 0.8'}resolution: {integrity: sha512-dZwN5L1VlUBewiP6H9s2+B3e3Jg96D0vzN+Ry73sOefebhYr9f94wwkMNN/9ouoU8pV1BqA1d1zGk8928cx0rg==}engines: {node: ^20.19.0 || >=22.12.0}hasBin: true@@ -13025,61 +13028,9 @@ packages:rollup: 4.54.0tinyglobby: 0.2.15tsx: 4.21.0- optionalDependencies:- fsevents: 2.3.3- dev: true-- resolution: {integrity: sha512-dZwN5L1VlUBewiP6H9s2+B3e3Jg96D0vzN+Ry73sOefebhYr9f94wwkMNN/9ouoU8pV1BqA1d1zGk8928cx0rg==}- engines: {node: ^20.19.0 || >=22.12.0}- hasBin: true- peerDependencies:- '@types/node': ^20.19.0 || >=22.12.0- jiti: '>=1.21.0'- less: ^4.0.0- lightningcss: ^1.21.0- sass: ^1.70.0- sass-embedded: ^1.70.0- stylus: '>=0.54.8'- sugarss: ^5.0.0- terser: ^5.16.0- tsx: ^4.8.1- yaml: ^2.4.2- peerDependenciesMeta:- '@types/node':- optional: true- jiti:- optional: true- less:- optional: true- lightningcss:- optional: true- sass:- optional: true- sass-embedded:- optional: true- stylus:- optional: true- sugarss:- optional: true- terser:- optional: true- tsx:- optional: true- yaml:- optional: true- dependencies:- '@types/node': 20.19.27- esbuild: 0.27.2- fdir: 6.5.0([email protected])- picomatch: 4.0.3- postcss: 8.5.6- rollup: 4.54.0- tinyglobby: 0.2.15yaml: 2.8.2optionalDependencies:fsevents: 2.3.3- dev: false/[email protected](@opentelemetry/[email protected])(@types/[email protected])(@vitest/[email protected])([email protected]):resolution: {integrity: sha512-flY6ScbCIt9HThs+C5HS7jvGOB560DJtk/Z15IQROTA6zEy49Nh8T/dofWTQL+n3vswqn87sbJNiuqw1SDp5Ig==}@@ -13143,7 +13094,7 @@ packages:tinyexec: 1.0.2tinyglobby: 0.2.15tinyrainbow: 3.1.0- vite: 7.3.0(@types/[email protected])([email protected])why-is-node-running: 2.3.0transitivePeerDependencies:- msw@@ -13211,7 +13162,7 @@ packages:tinyexec: 1.0.2tinyglobby: 0.2.15tinyrainbow: 3.1.0- vite: 7.3.0(@types/[email protected])([email protected])why-is-node-running: 2.3.0transitivePeerDependencies:- msw