Skip to content

armgit / ruflo

Public Imported from GitHub
0

🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated

test(mods): probe gate fails on lost calibration; --report splices below the marker

ruvbf8471283323
diff --git a/scripts/__tests__/probe-mod-guards.test.mjs b/scripts/__tests__/probe-mod-guards.test.mjs
index e30b755166..0db5f770d6 100644
--- a/scripts/__tests__/probe-mod-guards.test.mjs
+++ b/scripts/__tests__/probe-mod-guards.test.mjs
@@ -184,3 +184,24 @@ test('CLI contract: exit 2 on a bad flag, exit 3 when nothing is found', () => {
mkdirSync(empty)
assert.equal(run('--plugins-dir', empty).status, 3)
})
+
+test('a baselined guard that stops refusing anything fails instead of reading as fixed', () => {
+ const dir = fixtures(join(tmp, 'plugins-lost'), { naive: 'export function verdict(tool: string, input: unknown): string | undefined {\n return undefined\n}' })
+ const base = join(tmp, 'known-lost.json')
+ writeFileSync(base, JSON.stringify({ naive: ['nest-obj-12', 'key-name'] }))
+ const r = run('--fast', '--plugins-dir', dir, '--known-holes', base)
+ assert.equal(r.status, 1, r.stdout)
+ assert.match(r.stdout, /calibration lost/)
+ assert.doesNotMatch(r.stdout, /known holes that now pass/)
+})
+
+test('--report keeps the hand-written text above the generated marker', () => {
+ const dir = fixtures(join(tmp, 'plugins-report'), { screening: SCREENING })
+ const out = join(tmp, 'report.md')
+ writeFileSync(out, '# Findings\nhand written\n<!-- generated below: x -->\nold generated text\n')
+ assert.equal(run('--fast', '--plugins-dir', dir, '--report', out).status, 0)
+ const text = readFileSync(out, 'utf8')
+ assert.match(text, /^# Findings\nhand written\n<!-- generated below: x -->\n/)
+ assert.doesNotMatch(text, /old generated text/)
+ assert.match(text, /## Per-plugin summary/)
+})
diff --git a/scripts/probe-mod-guards.mjs b/scripts/probe-mod-guards.mjs
index 7566b7f671..706216fe27 100644
--- a/scripts/probe-mod-guards.mjs
+++ b/scripts/probe-mod-guards.mjs
@@ -12,7 +12,8 @@
// node scripts/probe-mod-guards.mjs --only ruflo-adr,ruflo-rvf --verbose
// node scripts/probe-mod-guards.mjs --known-holes scripts/probe-mod-guards.known-holes.json # only NEW holes fail
// node scripts/probe-mod-guards.mjs --write-known-holes <file> # record current holes
-// node scripts/probe-mod-guards.mjs --report v3/docs/validation/mod-guard-probe-2026-10.md # write the markdown report
+// node scripts/probe-mod-guards.mjs --report <path.md> # write the markdown report; if <path.md> already has a
+// # `<!-- generated below` marker, only the part below it is replaced
// node scripts/probe-mod-guards.mjs --strict # advisory (encoding/confusable) misses fail too
// node scripts/probe-mod-guards.mjs --plugins-dir <dir> # probe <dir>/*/hooks/guard.ts (used by the tests)
//
@@ -159,7 +160,13 @@ const isHole = r => r.status === 'fail' || (args.strict && r.status === 'info')
const holes = rows.flatMap(r => r.results.filter(isHole).map(x => `${r.name}:${x.id}`))
const known = args.knownHoles && existsSync(args.knownHoles) ? new Set(Object.entries(JSON.parse(readFileSync(args.knownHoles, 'utf8'))).flatMap(([p, ids]) => ids.map(i => `${p}:${i}`))) : new Set()
const fresh = holes.filter(h => !known.has(h))
-const ran = (k) => { const i = k.indexOf(':'); return rows.find(r => r.name === k.slice(0, i))?.results.some(x => x.id === k.slice(i + 1) && !isHole(x)) }
+// A baselined guard whose secret surface can no longer be found (it stopped refusing anything) must not read as 26 fixes: every evasion
+// probe is then skipped. It is a failure, and "fixed" needs an actual pass.
+for (const r of rows) {
+ const listed = [...known].filter(k => k.startsWith(`${r.name}:`)).length
+ if (listed > 0 && r.meta.calibrated === 0 && !r.fatal) r.fatal = `calibration lost: the baseline lists ${listed} holes but no secret surface was found (the guard stopped refusing secrets?)`
+}
+const ran = (k) => { const i = k.indexOf(':'); return rows.find(r => r.name === k.slice(0, i))?.results.some(x => x.id === k.slice(i + 1) && x.status === 'pass') }
const fixed = [...known].filter(ran)
const fatal = rows.filter(r => r.fatal)
@@ -171,7 +178,11 @@ if (args.writeKnownHoles) {
}
if (args.report) {
mkdirSync(dirname(args.report), { recursive: true })
- writeFileSync(args.report, renderReport({ rows, args: { fast: args.fast, budgetMs: args.budgetMs }, startedMs: started }))
+ const generated = renderReport({ rows, args: { fast: args.fast, budgetMs: args.budgetMs }, startedMs: started })
+ const MARK = '<!-- generated below'
+ const prior = existsSync(args.report) ? readFileSync(args.report, 'utf8') : ''
+ const at = prior.indexOf(MARK)
+ writeFileSync(args.report, at < 0 ? generated : `${prior.slice(0, prior.indexOf('\n', at) + 1)}\n${generated}`)
}
if (args.format === 'json') {
diff --git a/v3/docs/validation/mod-guard-probe-2026-10.md b/v3/docs/validation/mod-guard-probe-2026-10.md
index 5b8ffbf820..d1b53da0f7 100644
--- a/v3/docs/validation/mod-guard-probe-2026-10.md
+++ b/v3/docs/validation/mod-guard-probe-2026-10.md
@@ -100,49 +100,49 @@ print as "known holes that now pass"; remove them from the baseline. Fix a root
<!-- generated below: node scripts/probe-mod-guards.mjs --report <path> -->
-Generated by `node scripts/probe-mod-guards.mjs` in 17.4s. Timing gate: 50 ms per call (best of 3 runs).
+Generated by `node scripts/probe-mod-guards.mjs` in 16.5s. Timing gate: 50 ms per call (best of 3 runs).
## Per-plugin summary
| plugin | secret surface found (tool[namespace].field) | pass | fail | advisory miss | skipped | worst ms (probe) |
|---|---|---:|---:|---:|---:|---|
-| ruflo-adr | `memory_store[adr].value`<br>`agentdb_hierarchical-store[adr].value`<br>`agentdb_causal-edge[adr].value` | 49 | 26 | 6 | 0 | 80.8 (s-wide-keys-100k) |
-| ruflo-agent | `wasm_agent_prompt.value`<br>`wasm_agent_tool.value`<br>`wasm_agent_create.value` | 49 | 26 | 6 | 0 | 77.9 (s-wide-keys-100k) |
-| ruflo-agentdb | `memory_store.value`<br>`agentdb_hierarchical-store.value`<br>`agentdb_causal-edge.value` | 49 | 26 | 6 | 0 | 83.7 (s-wide-keys-100k) |
-| ruflo-ai-team | `task_create.value`<br>`memory_remember.value`<br>`task_update.value` | 49 | 26 | 6 | 0 | 82.3 (s-wide-keys-100k) |
-| ruflo-aidefence | `aidefence_learn.value` | 49 | 26 | 6 | 0 | 79.6 (s-wide-keys-100k) |
-| ruflo-autopilot | `autopilot_log.value`<br>`autopilot_learn.value`<br>`autopilot_config.value` | 49 | 26 | 6 | 0 | 90.2 (s-wide-keys-100k) |
-| ruflo-bbs-federation | `federation_bbs_publish.value`<br>`federation_bbs_human_join.value` | 49 | 26 | 6 | 0 | 184.2 (s-wide-keys-100k) |
-| ruflo-browser | none (policy guard) | 36 | 0 | 0 | 45 | 15.9 (s-zwsp) |
-| ruflo-business-pods | none (policy guard) | 36 | 0 | 0 | 45 | 1.4 (s-emoji) |
-| ruflo-chatgpt-federation | none (policy guard) | 36 | 0 | 0 | 45 | 0.4 (s-assign-spaces) |
-| ruflo-daa | `daa_knowledge_share.value`<br>`daa_x.value` | 49 | 26 | 6 | 0 | 91.7 (s-wide-keys-100k) |
-| ruflo-ddd | `memory_store[ddd-model].value`<br>`agentdb_hierarchical-store[ddd-model].value` | 49 | 26 | 6 | 0 | 79.3 (s-wide-keys-100k) |
-| ruflo-deepseek-harness | none (policy guard) | 36 | 0 | 0 | 45 | 0.0 (s-pem-spaces) |
-| ruflo-docs | `memory_store[docs].value`<br>`hooks_worker-dispatch[document].value` | 49 | 26 | 6 | 0 | 85.3 (s-wide-keys-100k) |
-| ruflo-federation | `memory_store[federation].value`<br>`federation_bbs_publish.value` | 49 | 26 | 6 | 0 | 85.8 (s-wide-keys-100k) |
-| ruflo-goals | `memory_store[goals].value`<br>`agentdb_hierarchical-store[goals].value`<br>`agentdb_pattern-store[goals].value` | 49 | 26 | 6 | 0 | 84.5 (s-wide-keys-100k) |
-| ruflo-graph-intelligence | `sublinear_solve.value`<br>`sublinear_x.value` | 49 | 26 | 6 | 0 | 79.1 (s-wide-keys-100k) |
-| ruflo-intelligence | `memory_store[patterns].value`<br>`hooks_intelligence_pattern-store.value`<br>`hooks_transfer.value` | 49 | 26 | 6 | 0 | 92.4 (s-wide-keys-100k) |
-| ruflo-iot-cognitum | `memory_store[iot-devices].value` | 49 | 26 | 6 | 0 | 87.1 (s-wide-keys-100k) |
-| ruflo-jujutsu | `github_pr_manage.value` | 49 | 26 | 6 | 0 | 86.7 (s-wide-keys-100k) |
-| ruflo-knowledge-graph | `agentdb_hierarchical-store.value`<br>`agentdb_causal-edge.value`<br>`agentdb_pattern-store.value` | 49 | 26 | 6 | 0 | 90.0 (s-wide-keys-100k) |
-| ruflo-loop-workers | `hooks_worker-dispatch.value` | 49 | 26 | 6 | 0 | 88.7 (s-wide-keys-100k) |
-| ruflo-market-data | `memory_store.value`<br>`agentdb_hierarchical-store.value`<br>`agentdb_pattern-store.value` | 49 | 26 | 6 | 0 | 91.1 (s-wide-keys-100k) |
-| ruflo-metaharness | `memory_store.value`<br>`agentdb_hierarchical-store.value`<br>`agentdb_causal-edge.value` | 49 | 26 | 6 | 0 | 85.0 (s-wide-keys-100k) |
-| ruflo-migrations | `memory_store.value`<br>`agentdb_hierarchical-store.value`<br>`agentdb_pattern-store.value` | 52 | 23 | 6 | 0 | 98.0 (s-wide-keys-100k) |
-| ruflo-music | `memory_store.value`<br>`agentdb_hierarchical-store.value`<br>`agentdb_pattern-store.value` | 52 | 23 | 6 | 0 | 95.0 (s-wide-keys-100k) |
-| ruflo-neural-trader | `memory_store.value`<br>`agentdb_hierarchical-store.value`<br>`agentdb_pattern-store.value` | 52 | 23 | 6 | 0 | 90.6 (s-wide-keys-100k) |
-| ruflo-observability | `memory_store.value`<br>`agentdb_hierarchical-store.value`<br>`agentdb_pattern-store.value` | 52 | 23 | 6 | 0 | 87.6 (s-wide-keys-100k) |
-| ruflo-rag-memory | `memory_store.value`<br>`agentdb_hierarchical-store.value`<br>`agentdb_pattern-store.value` | 49 | 26 | 6 | 0 | 92.0 (s-wide-keys-100k) |
-| ruflo-ruvector | `memory_store.value` | 49 | 26 | 6 | 0 | 84.8 (s-wide-keys-100k) |
-| ruflo-ruvllm | `ruvllm_hnsw_add.value` | 49 | 26 | 6 | 0 | 79.6 (s-wide-keys-100k) |
-| ruflo-rvf | `memory_store.value`<br>`hooks_transfer.value` | 49 | 26 | 6 | 0 | 92.3 (s-wide-keys-100k) |
-| ruflo-security-audit | `memory_store[security].value`<br>`agentdb_hierarchical-store[security].value`<br>`agentdb_pattern-store[security].value` | 49 | 26 | 6 | 0 | 81.3 (s-wide-keys-100k) |
-| ruflo-sparc | `memory_store[sparc].value`<br>`agentdb_hierarchical-store[sparc].value`<br>`agentdb_pattern-store[sparc].value` | 49 | 26 | 6 | 0 | 77.4 (s-wide-keys-100k) |
-| ruflo-testgen | none (policy guard) | 36 | 0 | 0 | 45 | 0.3 (s-deep-arr-5000) |
-| ruflo-workflows | `memory_store[workflows].value`<br>`agentdb_hierarchical-store[workflows].value`<br>`agentdb_pattern-store[workflows].value` | 49 | 26 | 6 | 0 | 60.5 (s-wide-keys-100k) |
-| ruflo-x-gateway | `x_federation_publish.value`<br>`x_federation_channel_publish.value` | 49 | 26 | 6 | 0 | 67.8 (s-wide-keys-100k) |
+| ruflo-adr | `memory_store[adr].value`<br>`agentdb_hierarchical-store[adr].value`<br>`agentdb_causal-edge[adr].value` | 49 | 26 | 6 | 0 | 86.6 (s-wide-keys-100k) |
+| ruflo-agent | `wasm_agent_prompt.value`<br>`wasm_agent_tool.value`<br>`wasm_agent_create.value` | 49 | 26 | 6 | 0 | 77.4 (s-wide-keys-100k) |
+| ruflo-agentdb | `memory_store.value`<br>`agentdb_hierarchical-store.value`<br>`agentdb_causal-edge.value` | 49 | 26 | 6 | 0 | 80.6 (s-wide-keys-100k) |
+| ruflo-ai-team | `task_create.value`<br>`memory_remember.value`<br>`task_update.value` | 49 | 26 | 6 | 0 | 92.6 (s-wide-keys-100k) |
+| ruflo-aidefence | `aidefence_learn.value` | 49 | 26 | 6 | 0 | 79.7 (s-wide-keys-100k) |
+| ruflo-autopilot | `autopilot_log.value`<br>`autopilot_learn.value`<br>`autopilot_config.value` | 49 | 26 | 6 | 0 | 77.1 (s-wide-keys-100k) |
+| ruflo-bbs-federation | `federation_bbs_publish.value`<br>`federation_bbs_human_join.value` | 49 | 26 | 6 | 0 | 169.9 (s-wide-keys-100k) |
+| ruflo-browser | none (policy guard) | 36 | 0 | 0 | 45 | 16.9 (s-zwsp) |
+| ruflo-business-pods | none (policy guard) | 36 | 0 | 0 | 45 | 3.9 (s-zwsp-in-secret) |
+| ruflo-chatgpt-federation | none (policy guard) | 36 | 0 | 0 | 45 | 0.4 (s-deep-arr-5000) |
+| ruflo-daa | `daa_knowledge_share.value`<br>`daa_x.value` | 49 | 26 | 6 | 0 | 81.6 (s-wide-keys-100k) |
+| ruflo-ddd | `memory_store[ddd-model].value`<br>`agentdb_hierarchical-store[ddd-model].value` | 49 | 26 | 6 | 0 | 72.3 (s-wide-keys-100k) |
+| ruflo-deepseek-harness | none (policy guard) | 36 | 0 | 0 | 45 | 0.6 (s-wide-array-100k) |
+| ruflo-docs | `memory_store[docs].value`<br>`hooks_worker-dispatch[document].value` | 49 | 26 | 6 | 0 | 77.6 (s-wide-keys-100k) |
+| ruflo-federation | `memory_store[federation].value`<br>`federation_bbs_publish.value` | 49 | 26 | 6 | 0 | 79.5 (s-wide-keys-100k) |
+| ruflo-goals | `memory_store[goals].value`<br>`agentdb_hierarchical-store[goals].value`<br>`agentdb_pattern-store[goals].value` | 49 | 26 | 6 | 0 | 74.9 (s-wide-keys-100k) |
+| ruflo-graph-intelligence | `sublinear_solve.value`<br>`sublinear_x.value` | 49 | 26 | 6 | 0 | 77.2 (s-wide-keys-100k) |
+| ruflo-intelligence | `memory_store[patterns].value`<br>`hooks_intelligence_pattern-store.value`<br>`hooks_transfer.value` | 49 | 26 | 6 | 0 | 71.3 (s-wide-keys-100k) |
+| ruflo-iot-cognitum | `memory_store[iot-devices].value` | 49 | 26 | 6 | 0 | 73.5 (s-wide-keys-100k) |
+| ruflo-jujutsu | `github_pr_manage.value` | 49 | 26 | 6 | 0 | 72.2 (s-wide-keys-100k) |
+| ruflo-knowledge-graph | `agentdb_hierarchical-store.value`<br>`agentdb_causal-edge.value`<br>`agentdb_pattern-store.value` | 49 | 26 | 6 | 0 | 79.7 (s-wide-keys-100k) |
+| ruflo-loop-workers | `hooks_worker-dispatch.value` | 49 | 26 | 6 | 0 | 79.2 (s-wide-keys-100k) |
+| ruflo-market-data | `memory_store.value`<br>`agentdb_hierarchical-store.value`<br>`agentdb_pattern-store.value` | 49 | 26 | 6 | 0 | 73.5 (s-wide-keys-100k) |
+| ruflo-metaharness | `memory_store.value`<br>`agentdb_hierarchical-store.value`<br>`agentdb_causal-edge.value` | 49 | 26 | 6 | 0 | 79.2 (s-wide-keys-100k) |
+| ruflo-migrations | `memory_store.value`<br>`agentdb_hierarchical-store.value`<br>`agentdb_pattern-store.value` | 52 | 23 | 6 | 0 | 91.4 (s-wide-keys-100k) |
+| ruflo-music | `memory_store.value`<br>`agentdb_hierarchical-store.value`<br>`agentdb_pattern-store.value` | 52 | 23 | 6 | 0 | 95.5 (s-wide-keys-100k) |
+| ruflo-neural-trader | `memory_store.value`<br>`agentdb_hierarchical-store.value`<br>`agentdb_pattern-store.value` | 52 | 23 | 6 | 0 | 97.4 (s-wide-keys-100k) |
+| ruflo-observability | `memory_store.value`<br>`agentdb_hierarchical-store.value`<br>`agentdb_pattern-store.value` | 52 | 23 | 6 | 0 | 86.6 (s-wide-keys-100k) |
+| ruflo-rag-memory | `memory_store.value`<br>`agentdb_hierarchical-store.value`<br>`agentdb_pattern-store.value` | 49 | 26 | 6 | 0 | 83.5 (s-wide-keys-100k) |
+| ruflo-ruvector | `memory_store.value` | 49 | 26 | 6 | 0 | 76.3 (s-wide-keys-100k) |
+| ruflo-ruvllm | `ruvllm_hnsw_add.value` | 49 | 26 | 6 | 0 | 77.3 (s-wide-keys-100k) |
+| ruflo-rvf | `memory_store.value`<br>`hooks_transfer.value` | 49 | 26 | 6 | 0 | 76.0 (s-wide-keys-100k) |
+| ruflo-security-audit | `memory_store[security].value`<br>`agentdb_hierarchical-store[security].value`<br>`agentdb_pattern-store[security].value` | 49 | 26 | 6 | 0 | 73.1 (s-wide-keys-100k) |
+| ruflo-sparc | `memory_store[sparc].value`<br>`agentdb_hierarchical-store[sparc].value`<br>`agentdb_pattern-store[sparc].value` | 49 | 26 | 6 | 0 | 72.9 (s-wide-keys-100k) |
+| ruflo-testgen | none (policy guard) | 36 | 0 | 0 | 45 | 0.0 (s-assign-long-value) |
+| ruflo-workflows | `memory_store[workflows].value`<br>`agentdb_hierarchical-store[workflows].value`<br>`agentdb_pattern-store[workflows].value` | 49 | 26 | 6 | 0 | 72.0 (s-wide-keys-100k) |
+| ruflo-x-gateway | `x_federation_publish.value`<br>`x_federation_channel_publish.value` | 49 | 26 | 6 | 0 | 80.7 (s-wide-keys-100k) |
## Probe matrix (✓ pass · ✗ fail · ~ advisory miss · · not applicable)