🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated
TypeScript agentic-aiagentic-frameworkagentic-workflowagentsai-agentsai-assistantai-skillsautonomous-agentsclaudecodexdsh-pluginharnessmcp-servermulti-agentmulti-agent-systemsnpmskillsswarmswarm-intelligencetypescript
test(mods): live headless proof of ruflo-mods route, deny, heartbeat and /ruflo-mods paths
ruv41be0939267f
diff --git a/scripts/live-ruflo-mods-paths.sh b/scripts/live-ruflo-mods-paths.shnew file mode 100755index 0000000000..5c20a5ff79--- /dev/null+++ b/scripts/live-ruflo-mods-paths.sh@@ -0,0 +1,129 @@+#!/usr/bin/env bash+# Live proof of the ruflo-mods paths the capability review marks "no" (v3/docs/validation/mod-capability-review-2026-10.md):+# prompt.submit routing context, a tool.check deny, the session heartbeat file, and the /ruflo-mods command, each against a REAL+# headless Claude Code session with the plugin loaded by --plugin-dir. Method follows scripts/live-ruflo-mods.sh.+# scripts/live-ruflo-mods-paths.sh [route deny status command | all] (default: all)+# Everything runs in private mktemp scratch projects. Spend: haiku only (about $1 for `all`). Env: LIVE_MODEL.+# The deny probe uses `echo <entry> > ran.txt` for a dangerous-list entry that is inert on Linux: a block leaves no ran.txt.+# Output is screened: a reply that looks like auth/account/telemetry content is withheld and counted, never printed or stored.+set -u+REPO="$(cd "$(dirname "$0")/.." && pwd)"+S="${LIVE_SCRATCH:-$(mktemp -d "${TMPDIR:-/tmp}/mods-paths.XXXXXX")}"+PHASES=("$@"); [ ${#PHASES[@]} -eq 0 ] && PHASES=(all)+[ "${PHASES[0]}" = all ] && PHASES=(route deny status command)+mkdir -p "$S"+echo "scratch: $S"+sed -n '/^# --- node driver ---$/,$p' "$0" | sed 1d >"$S/live.mjs"+for p in "${PHASES[@]}"; do echo "== phase: $p"; node "$S/live.mjs" "$p" "$S" "$REPO" || echo "phase $p exited $?"; done+echo "results: $S"+exit 0+# --- node driver ---+import { spawn, execFileSync } from 'node:child_process'+import { writeFileSync, readFileSync, mkdirSync, existsSync } from 'node:fs'+import { join } from 'node:path'++const [phase, S, REPO] = process.argv.slice(2)+const PLUG = `${REPO}/plugins/ruflo-mods`+const MODEL = process.env.LIVE_MODEL ?? 'haiku'+const AUTHLIKE = /bearer\s|authorization|api[_-]?key|sk-ant|oauth|access[_-]?token|refresh[_-]?token|account[_ -]?(id|uuid)|organization|[\w.+-]+@[\w-]+\.[a-z]{2,}/i+let withheld = 0+const clean = text => (AUTHLIKE.test(text ?? '') ? (withheld++, '[withheld: auth-like content]') : text)+const DANGER = 'format c:' // a DANGEROUS_COMMANDS entry that does nothing on Linux++// A fresh scratch project per session: `.claude-flow/` is the opt-in boundary the mod checks.+function project(name, { claudeFlow = true } = {}) {+ const dir = join(S, name)+ mkdirSync(dir, { recursive: true })+ execFileSync('git', ['init', '-q'], { cwd: dir })+ if (claudeFlow) mkdirSync(join(dir, '.claude-flow'), { recursive: true })+ return dir+}+const settingsFile = (dir, options) => {+ const f = join(S, `settings-${Math.random().toString(36).slice(2, 8)}.json`)+ const o = options ? { options } : undefined+ writeFileSync(f, JSON.stringify(o ? { pluginConfigs: { 'ruflo-mods@inline': o, 'ruflo-mods': o } } : {}))+ return f+}++async function session(dir, prompts, { plugin = true, options, tools = [], budget = '0.30' } = {}) {+ const denied = ['Read', 'Grep', 'Glob', 'WebFetch', 'WebSearch', 'Edit', 'Write', 'NotebookEdit', ...(tools.includes('Bash') ? [] : ['Bash'])]+ const args = ['-p', '--input-format', 'stream-json', '--output-format', 'stream-json', '--verbose', '--model', MODEL,+ '--settings', settingsFile(dir, options), '--setting-sources', 'local', '--max-budget-usd', budget,+ '--allowedTools', tools.join(',') || 'ToolSearch', '--disallowedTools', denied.join(','),+ '--append-system-prompt', 'Answer exactly as asked, no commentary.']+ if (plugin) args.push('--plugin-dir', PLUG)+ const p = spawn('claude', args, { cwd: dir, stdio: ['pipe', 'pipe', 'ignore'], env: { ...process.env, CLAUDE_CODE_ENABLE_FUNCTION_HOOKS: '1' } })+ let buf = '', onResult, init+ const toolRows = []+ p.stdout.on('data', d => {+ buf += d+ let i+ while ((i = buf.indexOf('\n')) >= 0) {+ const line = buf.slice(0, i); buf = buf.slice(i + 1)+ try {+ const m = JSON.parse(line)+ if (m.type === 'system' && m.subtype === 'init') init = m+ if (m.type === 'result' && onResult) onResult(m)+ for (const c of Array.isArray(m.message?.content) ? m.message.content : []) {+ if (c.type === 'tool_use') toolRows.push({ use: c.name, input: c.input?.command ?? '' })+ if (c.type === 'tool_result') toolRows.push({ result: typeof c.content === 'string' ? c.content : JSON.stringify(c.content), error: !!c.is_error })+ }+ } catch {}+ }+ })+ const rows = []+ for (const text of prompts) {+ const done = new Promise(res => { onResult = res })+ p.stdin.write(`${JSON.stringify({ type: 'user', message: { role: 'user', content: text } })}\n`)+ const r = await Promise.race([done, new Promise(res => setTimeout(() => res(undefined), 120_000))])+ rows.push({ prompt: text, reply: r?.result ?? '(timeout)', cost: r?.total_cost_usd })+ }+ p.stdin.end()+ await new Promise(res => { const t = setTimeout(() => (p.kill(), res()), 15_000); p.on('close', () => (clearTimeout(t), res())) })+ const loaded = (init?.plugins ?? []).map(x => x.name ?? String(x)).filter(n => /ruflo-mods/.test(n))+ return { rows, toolRows, loaded, cost: rows.at(-1)?.cost }+}+const show = (name, res) => {+ console.log(`[${name}] plugin loaded: ${res.loaded.length ? res.loaded.join(',') : 'no'}`)+ for (const r of res.rows) console.log(`[${name}] > ${r.prompt.slice(0, 110)}\n${clean(r.reply)}`)+ for (const t of res.toolRows) console.log(`[${name}] tool ${t.use ? `call: ${clean(`${t.use} ${t.input}`)}` : `${t.error ? 'ERROR ' : 'result: '}${clean(t.result).slice(0, 300)}`}`)+ console.log(`[${name}] cost $${res.cost}\n`)+}+const ROUTE = text => `Look in your context for a block that starts with the line "[INFO] Routing task:". If present, copy the whole block verbatim (the INFO line, Agent, Confidence, Reason lines) and nothing else. If there is no such block, reply exactly NONE. The task for this turn: ${text}`+const RUN = (cmd, ending) => `Run this exact shell command with the Bash tool and then reply ${ending}: ${cmd}`+const BAD = RUN(`echo ${DANGER} > ran.txt`, 'with only the word RAN, or with DENIED: <error text> if the call was refused')+const GOOD = RUN('echo hello > ran.txt', 'with only the word RAN, or with DENIED: <error text> if the call was refused')++async function route() {+ const T = 'write a unit test for the parser'+ for (const [name, task, o] of [['route-on-keyword', T], ['route-on-nomatch', 'zzz qqq'], ['route-ranked-block-off', T, { routeContext: false }]]) {+ show(name, await session(project(name), [ROUTE(task)], { options: o }))+ }+ show('route-no-plugin', await session(project('route-no-plugin'), [ROUTE(T)], { plugin: false }))+}+async function deny() {+ for (const [name, prompt, plugin] of [['deny-on-dangerous', BAD, true], ['deny-on-benign', GOOD, true], ['deny-control-no-plugin', BAD, false]]) {+ const d = project(name)+ show(name, await session(d, [prompt], { plugin, tools: ['Bash'] }))+ const f = join(d, 'ran.txt')+ console.log(`[${name}] ran.txt after the session: ${existsSync(f) ? `exists: ${JSON.stringify(readFileSync(f, 'utf8').trim())}` : 'absent'}\n`)+ }+}+async function status() { // the heartbeat the mod writes for `ruflo mods doctor`; no model call (the mod answers /ruflo-mods)+ const d = project('status-project')+ const file = join(d, '.claude-flow/mods/session.json')+ console.log(`[status-project] before: ${existsSync(file) ? 'exists' : 'absent'}`)+ show('status-project', await session(d, ['/ruflo-mods']))+ console.log(`[status-project] .claude-flow/mods/session.json: ${existsSync(file) ? readFileSync(file, 'utf8').replace(/\s+/g, ' ') : 'absent'}\n`)+ const bare = project('status-no-opt-in', { claudeFlow: false })+ show('status-no-opt-in', await session(bare, ['/ruflo-mods']))+ console.log(`[status-no-opt-in] .claude-flow created: ${existsSync(join(bare, '.claude-flow')) ? 'yes' : 'no'}\n`)+}+async function command() { // counters read by /ruflo-mods in the same session, before and after a routed prompt and a refused call+ show('command-counters', await session(project('command-counters'), ['/ruflo-mods', ROUTE('write a unit test for the parser'), BAD, '/ruflo-mods'], { tools: ['Bash'], budget: '0.40' }))+}+const PHASE = { route, deny, status, command }+if (!PHASE[phase]) { console.error(`unknown phase ${phase}`); process.exit(2) }+await PHASE[phase]()+console.log(`replies withheld as auth-like: ${withheld}`)+process.exit(0)diff --git a/v3/docs/validation/ruflo-mods-live-paths-2026-10.md b/v3/docs/validation/ruflo-mods-live-paths-2026-10.mdnew file mode 100644index 0000000000..33046e3814--- /dev/null+++ b/v3/docs/validation/ruflo-mods-live-paths-2026-10.md@@ -0,0 +1,65 @@+# ruflo-mods live paths: evidence, 2026-10-05++Closes review improvement #14 of `mod-capability-review-2026-10.md`: the ruflo-mods rows whose "live-verified" column read **no**.+Method: `scripts/live-ruflo-mods-paths.sh` (new; follows `scripts/live-ruflo-mods.sh`). Each probe is a real headless+`claude -p` session (model `haiku`, `--plugin-dir plugins/ruflo-mods`, `--setting-sources local`, `CLAUDE_CODE_ENABLE_FUNCTION_HOOKS=1`)+in its own `mktemp -d` git project with a `.claude-flow/` directory (the mod's opt-in boundary). Nothing under `plugins/` was changed.+Spend about $0.5 for all four phases (each probe $0.02 to $0.06; the `/ruflo-mods` probes cost $0). Outputs below are copied from the runs.++Auth screening: every reply and tool result is matched against an auth/account/telemetry pattern before printing. Replies withheld: **0**.+Nothing auth-like appeared, so nothing was omitted; the engine's init message is never stored beyond the plugin names.++## 1. Routing: `prompt.submit` route context (proven)++Prompt asks the model to copy the `[INFO] Routing task:` block from its context, or reply `NONE`.++| Probe | Observed |+|---|---|+| plugin loaded, "write a unit test for the parser" | block with `Agent: tester`, `Confidence: 60.0%`, `Reason: Matched keyword(s) from: test\|tests\|testing\|spec\|spe` |+| plugin loaded, "zzz qqq" (no keyword) | block with `Agent: coder`, `Confidence: 30.0%`, `Reason: Default routing - no specific keyword matched` (the #3567 no-match wording) |+| plugin loaded, `routeContext: false` | the same tester block, so that option does **not** turn routing off; it only drops the ranked-memory block (see its description) |+| **no plugin** (control) | `NONE` |++So the in-process route reaches the model as prompt context, and without the mod the block is absent. The scratch project has no classic+hook configured, so the mod owned `route` (status below shows `owns: route, post-edit`).++## 2. Deny: `tool.check` dangerous-command list (proven)++The probe asks for `echo format c: > ran.txt`, a `DANGEROUS_COMMANDS` entry that is inert on Linux, so a permitted run is harmless+and `ran.txt` shows whether the command executed.++| Probe | Tool result the model saw | `ran.txt` |+|---|---|---|+| plugin, dangerous | `ERROR Permission to use Bash denied by plugin ruflo-mods: ruflo: dangerous command blocked (format c:)`; model replied `DENIED: ruflo: dangerous command blocked (format c:)` | absent |+| plugin, benign `echo hello > ran.txt` | `(Bash completed with no output)`; model replied `RAN` | `hello` |+| **no plugin** (control), dangerous | `(Bash completed with no output)`; model replied `RAN` | `format c:` |++The deny is the mod's: same command, same project shape, only the plugin differs; the benign command is not caught.++## 3. Status file: the session heartbeat (proven; the path is `session.json`)++There is no `status.json`. The mod's file is `.claude-flow/mods/session.json` (`hooks/session.ts:14`, display only, read by `ruflo mods doctor`).++- Project with `.claude-flow/`: before the session `absent`; after: `{ "startedAt": "2026-10-05T05:56:00.886Z", "owned": [ "route", "post-edit" ], "statusLine": true }`.+- Project without `.claude-flow/`: `owns: nothing (classic hooks keep every event)` and `.claude-flow` **not created** (the opt-in boundary holds).++## 4. `/ruflo-mods` command (proven, answered by the mod with no model call, cost $0)++One session, in order: `/ruflo-mods`, a routed prompt, a refused Bash call, `/ruflo-mods`.++First report: `routed: 0 prompt(s); last none yet`, `policy: none; 0 call(s) tightened, 0 observed`.+Second report: `routed: 2 prompt(s); last coder (30%, no match)`, `policy: none; 1 call(s) tightened, 0 observed`.+Two prompts were routed (the route prompt and the Bash prompt, which matches no keyword), and one call was tightened: the counters+agree with what the session did. The full report also lists `owns`, `edits`, `budget`, `tool hints`, `agent trim`, `delivery`, `segments`, `guidance` (all `off`/`none` by default).++## What could not be provoked, or is not proven++- **Route rationale quoting**: the model sometimes copies the block's first line as `[INFO] Routing task:` followed by part of the prompt, and+ sometimes drops the prompt text; the box lines (agent, confidence, reason) were stable across runs. Proof rests on those lines plus the `NONE` control.+- **Routing off switch**: no option disables routing alone, so there is no "mod loaded, routing off" control; the control is "no plugin".+- **`rm -rf /` and the other list entries**: not run live. `rm -rf /` is destructive if the mod failed to load, so only the inert `format c:` entry was used; the other entries are covered by the parity unit tests.+- **Policy projection deny/ask** (`.claude-flow/policy/claude-code.json`): not exercised; `policy: none` throughout.+- **Edit learning, cost ladder, guidance observation, trust gate** rows of the matrix: out of scope for this item.+- Single run per probe at `haiku`; a different model may phrase its quote differently. The deny and status results are deterministic (hook output and file state), the route quote is model-mediated.++Reproduce: `scripts/live-ruflo-mods-paths.sh all` (or one of `route deny status command`).