Skip to content

armgit / ruflo

Public Imported from GitHub
0

🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated

feat(cost-tracker): P3 β€” auto-emit hooks_model-outcome (v0.7.0)

Reuven333e6338d9c3
diff --git a/plugins/ruflo-cost-tracker/.claude-plugin/plugin.json b/plugins/ruflo-cost-tracker/.claude-plugin/plugin.json
index 74f07e19f8..defb26f01b 100644
--- a/plugins/ruflo-cost-tracker/.claude-plugin/plugin.json
+++ b/plugins/ruflo-cost-tracker/.claude-plugin/plugin.json
@@ -1,7 +1,7 @@
{
"name": "ruflo-cost-tracker",
"description": "Token usage tracking, model cost attribution per agent, budget alerts, and optimization recommendations β€” uses memory_* (namespace-routed) for cost-tracking and cost-patterns; pairs with federation budget circuit breaker (ADR-097)",
- "version": "0.6.0",
+ "version": "0.7.0",
"author": {
"name": "ruvnet",
"url": "https://github.com/ruvnet"
@@ -24,6 +24,7 @@
"benchmarking",
"verified",
"telemetry",
- "budget"
+ "budget",
+ "model-outcome"
]
}
diff --git a/plugins/ruflo-cost-tracker/README.md b/plugins/ruflo-cost-tracker/README.md
index 20f76f3e73..502358f0e4 100644
--- a/plugins/ruflo-cost-tracker/README.md
+++ b/plugins/ruflo-cost-tracker/README.md
@@ -41,7 +41,8 @@ cost budget check [--period ...] # Compute utilization + alert; exit 1
cost report [--period today|week|month] # Cost report (with By-tier block, reads measured booster data)
cost breakdown [--by agent|model|task] # Detailed breakdown by dimension
cost budget set <amount> # Set budget limit in USD
-cost optimize # Analyze usage and suggest savings (+ hooks_model-outcome feedback)
+cost optimize # Analyze usage and suggest savings (+ auto-emits hooks_model-outcome via outcome.mjs)
+cost outcome <task> <model> <outcome> # Emit hooks_model-outcome (success|escalated|failure) so the router learns
cost benchmark [--llm] [--anthropic] # Run measured benchmark β€” booster + optional Gemini/Sonnet/Opus baselines
cost workers # Inspect optimize + benchmark loop-workers consumed
cost history # Show cost tracking over time
@@ -130,7 +131,7 @@ Reserved namespaces (`pattern`, `claude-memories`, `default`) MUST NOT be shadow
```bash
bash plugins/ruflo-cost-tracker/scripts/smoke.sh
-# Expected: "33 passed, 0 failed"
+# Expected: "36 passed, 0 failed"
```
## Architecture Decisions
diff --git a/plugins/ruflo-cost-tracker/commands/ruflo-cost.md b/plugins/ruflo-cost-tracker/commands/ruflo-cost.md
index 865b17b92d..983b8b89c8 100644
--- a/plugins/ruflo-cost-tracker/commands/ruflo-cost.md
+++ b/plugins/ruflo-cost-tracker/commands/ruflo-cost.md
@@ -49,6 +49,12 @@ Cost tracking commands:
2. Print: total cost, per-model and per-tier breakdown, persisted memory key
3. Sets the `cost-tracking` namespace record at key `session-<sessionId>` (consumed by `cost-report` step 1)
+**`cost outcome <task> <model> <outcome>`** -- Emit a `hooks_model-outcome` event so the router learns from applied recommendations. Auto-wired into `cost-optimize` step 8.
+1. Validates `outcome ∈ {success, escalated, failure}`
+2. Runs `node plugins/ruflo-cost-tracker/scripts/outcome.mjs "<task>" <model> <outcome>`
+3. The script wraps `npx @claude-flow/cli hooks model-outcome -t ... -m ... -o ...` with explicit-argv spawnSync so quoting is safe
+4. Without this, the router doesn't learn from cost-optimize recommendations and the Tier 1 bypass rate doesn't tighten over time
+
**`cost benchmark [--llm] [--anthropic]`** -- Run the corpus benchmark to verify booster claims with measured numbers.
1. Without flags: booster-only (free, ~85 ms wall-time, no API keys needed)
2. `--llm`: also run Gemini 2.0 Flash baseline (uses GCP `GOOGLE_AI_API_KEY` secret)
diff --git a/plugins/ruflo-cost-tracker/scripts/outcome.mjs b/plugins/ruflo-cost-tracker/scripts/outcome.mjs
new file mode 100755
index 0000000000..cb17ccab74
--- /dev/null
+++ b/plugins/ruflo-cost-tracker/scripts/outcome.mjs
@@ -0,0 +1,43 @@
+#!/usr/bin/env node
+// cost-outcome β€” thin wrapper around `hooks model-outcome` so cost-optimize
+// recommendations can be emitted as pre-formatted shell commands. The agent
+// (or human) runs this once per applied recommendation and the router learns.
+//
+// Usage:
+// node scripts/outcome.mjs <task-description> <model> <outcome>
+// outcome ∈ { success, escalated, failure }
+//
+// Example:
+// node scripts/outcome.mjs "format imports" haiku success
+// node scripts/outcome.mjs "design auth flow" sonnet escalated
+//
+// Why a wrapper: `hooks model-outcome` takes `-t -m -o` and quoting through
+// shell pipelines is error-prone. This script uses spawnSync with explicit
+// argv so a task description with quotes/newlines is preserved.
+
+import { spawnSync } from 'node:child_process';
+
+const ALLOWED = new Set(['success', 'escalated', 'failure']);
+
+function main() {
+ const [task, model, outcome] = process.argv.slice(2);
+ if (!task || !model || !outcome) {
+ console.error('usage: outcome.mjs <task-description> <model> <outcome>');
+ console.error(' outcome ∈ {success, escalated, failure}');
+ process.exit(2);
+ }
+ if (!ALLOWED.has(outcome)) {
+ console.error(`invalid outcome '${outcome}' β€” must be one of: success, escalated, failure`);
+ process.exit(2);
+ }
+ const r = spawnSync('npx', [
+ '@claude-flow/cli@latest', 'hooks', 'model-outcome',
+ '-t', task, '-m', model, '-o', outcome,
+ ], { stdio: 'inherit' });
+ if (r.status !== 0) {
+ console.error(`emit failed (exit ${r.status})`);
+ process.exit(r.status || 1);
+ }
+}
+
+main();
diff --git a/plugins/ruflo-cost-tracker/scripts/smoke.sh b/plugins/ruflo-cost-tracker/scripts/smoke.sh
index 95a79d65ed..f6d31d7c33 100755
--- a/plugins/ruflo-cost-tracker/scripts/smoke.sh
+++ b/plugins/ruflo-cost-tracker/scripts/smoke.sh
@@ -8,10 +8,10 @@ step() { printf "β†’ %s ... " "$1"; }
ok() { printf "PASS\n"; PASS=$((PASS+1)); }
bad() { printf "FAIL: %s\n" "$1"; FAIL=$((FAIL+1)); }
-step "1. plugin.json declares 0.6.0 with new keywords"
+step "1. plugin.json declares 0.7.0 with new keywords"
v=$(grep -E '"version"' "$ROOT/.claude-plugin/plugin.json" | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1)
-if [[ "$v" != "0.6.0" ]]; then
- bad "expected 0.6.0, got '$v'"
+if [[ "$v" != "0.7.0" ]]; then
+ bad "expected 0.7.0, got '$v'"
else
miss=""
for k in namespace-routing mcp agentic-flow agent-booster tier1-routing model-routing benchmarking verified telemetry budget; do
@@ -271,5 +271,24 @@ grep -q "cost budget check" "$F" || miss="$miss check"
grep -qE "50/75/90/100|alert ladder|HARD_STOP" "$F" || miss="$miss alert-ladder"
[[ -z "$miss" ]] && ok || bad "$miss"
+step "34. cost-optimize step 8 wires auto-emit via outcome.mjs"
+F="$ROOT/skills/cost-optimize/SKILL.md"
+grep -q "outcome\.mjs" "$F" && grep -qE "success|escalated|failure" "$F" \
+ && ok || bad "step 8 not wired to outcome.mjs"
+
+step "35. outcome.mjs harness present + parses + validates outcomes"
+F="$ROOT/scripts/outcome.mjs"
+miss=""
+[[ -x "$F" ]] || miss="$miss not-executable"
+node --check "$F" 2>/dev/null || miss="$miss syntax-error"
+grep -q "spawnSync" "$F" || miss="$miss no-spawnSync"
+grep -q "hooks.*model-outcome" "$F" || miss="$miss no-hooks-call"
+grep -qE "success.*escalated.*failure|ALLOWED" "$F" || miss="$miss no-validation"
+[[ -z "$miss" ]] && ok || bad "$miss"
+
+step "36. ruflo-cost.md documents 'cost outcome' subcommand"
+grep -q "cost outcome" "$ROOT/commands/ruflo-cost.md" \
+ && ok || bad "missing"
+
printf "\n%s passed, %s failed\n" "$PASS" "$FAIL"
[[ $FAIL -eq 0 ]] || exit 1
diff --git a/plugins/ruflo-cost-tracker/skills/cost-optimize/SKILL.md b/plugins/ruflo-cost-tracker/skills/cost-optimize/SKILL.md
index c6cf0491bc..74f13e3d89 100644
--- a/plugins/ruflo-cost-tracker/skills/cost-optimize/SKILL.md
+++ b/plugins/ruflo-cost-tracker/skills/cost-optimize/SKILL.md
@@ -26,7 +26,17 @@ When costs are higher than expected or you want to proactively reduce spending.
7. **Store the optimization pattern** -- two paths:
- **Pattern store (typed, recommended)**: `mcp__claude-flow__agentdb_pattern-store` with `type: 'cost-optimization'`. Don't pass a `namespace` arg β€” ReasoningBank routes it; on bridge unavailability the fallback writes to the reserved `pattern` namespace with `controller: 'memory-store-fallback'` (see ruflo-agentdb ADR-0001).
- **Plain store (namespace-routable)**: `mcp__claude-flow__memory_store --namespace cost-patterns` β€” this DOES respect the `cost-patterns` namespace because `memory_*` is namespace-routed.
-8. **Close the routing feedback loop** -- when a downgrade recommendation is *applied* (or its application is logged elsewhere in this session), call `mcp__claude-flow__hooks_model-outcome` with `{task, fromModel, toModel, outcome: 'success'|'escalated'|'failure'}`. This is the typed equivalent of the legacy `routing-outcomes` namespace (see ruflo-intelligence ADR-0001 Β§"Neutral"). Without this signal the router does not learn from cost-tracker's recommendations and the booster bypass rate (see `cost-booster-route` skill) does not improve over time.
+8. **Close the routing feedback loop β€” auto-emit `hooks_model-outcome`** -- for each downgrade recommendation, format the outcome-emit command as part of the recommendation table so it can be run directly:
+
+ ```bash
+ # success path (downgrade worked)
+ node plugins/ruflo-cost-tracker/scripts/outcome.mjs "<task-description>" <model> success
+
+ # escalated path (had to upgrade after downgrade attempt)
+ node plugins/ruflo-cost-tracker/scripts/outcome.mjs "<task-description>" <model> escalated
+ ```
+
+ The script wraps `npx @claude-flow/cli hooks model-outcome -t ... -m ... -o ...` with explicit-argv `spawnSync` so quoting is safe. Without this signal the router does not learn from cost-tracker's recommendations and the booster bypass rate (see `cost-booster-route` skill) does not improve over time. This is the typed equivalent of the legacy `routing-outcomes` namespace (see ruflo-intelligence ADR-0001 Β§"Neutral").
9. **Report** -- display: ranked recommendations with savings estimate, total potential savings, implementation priority (quick wins first), and any model-outcome events emitted in step 8
## CLI alternative