Skip to content

armgit / ruflo

Public Imported from GitHub
0

🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated

feat(cost-tracker): P8 β€” GitHub Actions smoke + booster bench on PR (v0.11.0)

Reuven4b3169df91dc
diff --git a/.github/workflows/cost-tracker-smoke.yml b/.github/workflows/cost-tracker-smoke.yml
new file mode 100644
index 0000000000..dca9145b0d
--- /dev/null
+++ b/.github/workflows/cost-tracker-smoke.yml
@@ -0,0 +1,73 @@
+# Smoke + booster-only bench for plugins/ruflo-cost-tracker.
+#
+# Triggers on changes to the plugin or its corpus. Smoke is fast (~100 ms,
+# pure bash + node --check) so it always runs. The booster-only bench runs
+# locally β€” installs `agent-booster` in a sibling temp dir then invokes
+# bench.mjs from there so node-resolve picks up the package. The LLM and
+# Anthropic baselines are intentionally OMITTED: they cost real money per
+# run and require Secret Manager keys; they belong in a manual-trigger or
+# scheduled workflow with a budget guard, not on every PR.
+name: cost-tracker-smoke
+
+on:
+ push:
+ branches: [main]
+ paths:
+ - 'plugins/ruflo-cost-tracker/**'
+ - '.github/workflows/cost-tracker-smoke.yml'
+ pull_request:
+ paths:
+ - 'plugins/ruflo-cost-tracker/**'
+ - '.github/workflows/cost-tracker-smoke.yml'
+ workflow_dispatch:
+
+jobs:
+ smoke:
+ runs-on: ubuntu-latest
+ timeout-minutes: 8
+ steps:
+ - uses: actions/checkout@v4
+
+ - uses: actions/setup-node@v4
+ with:
+ node-version: '20'
+
+ - name: Run smoke (39+ structural checks)
+ run: bash plugins/ruflo-cost-tracker/scripts/smoke.sh
+
+ - name: Install agent-booster for the bench
+ run: |
+ mkdir -p .ci-bench
+ cd .ci-bench
+ npm init -y > /dev/null
+ # Pin to the same major as v3/node_modules to keep results comparable.
+ npm install --no-audit --no-fund --silent agent-booster@^0.2
+
+ - name: Run booster-only bench (no LLM cost)
+ run: |
+ cd .ci-bench
+ node ../plugins/ruflo-cost-tracker/scripts/bench.mjs
+
+ - name: Trend report (drift across runs in this checkout)
+ run: node plugins/ruflo-cost-tracker/scripts/trend.mjs
+ # The checkout only contains the runs that were committed β€” useful
+ # as a sanity check that trend.mjs runs cleanly on real data.
+
+ - name: Verify Tier 1 win rate β‰₯ 0.80 (regression gate)
+ run: |
+ node -e "
+ const d = JSON.parse(require('fs').readFileSync('plugins/ruflo-cost-tracker/docs/benchmarks/runs/latest.json'));
+ if (d.summary.winRate < 0.80) {
+ console.error('REGRESSION: Tier 1 win rate', d.summary.winRate, '< 0.80');
+ process.exit(1);
+ }
+ console.log('Tier 1 win rate:', (d.summary.winRate * 100).toFixed(1) + '%');
+ "
+
+ - name: Upload bench artifact
+ if: always()
+ uses: actions/upload-artifact@v4
+ with:
+ name: cost-tracker-bench-result
+ path: plugins/ruflo-cost-tracker/docs/benchmarks/runs/latest.json
+ retention-days: 30
diff --git a/plugins/ruflo-cost-tracker/.claude-plugin/plugin.json b/plugins/ruflo-cost-tracker/.claude-plugin/plugin.json
index 8c8094f02f..b2f56e1e76 100644
--- a/plugins/ruflo-cost-tracker/.claude-plugin/plugin.json
+++ b/plugins/ruflo-cost-tracker/.claude-plugin/plugin.json
@@ -1,7 +1,7 @@
{
"name": "ruflo-cost-tracker",
"description": "Token usage tracking, model cost attribution per agent, budget alerts, and optimization recommendations β€” uses memory_* (namespace-routed) for cost-tracking and cost-patterns; pairs with federation budget circuit breaker (ADR-097)",
- "version": "0.10.0",
+ "version": "0.11.0",
"author": {
"name": "ruvnet",
"url": "https://github.com/ruvnet"
@@ -26,6 +26,7 @@
"telemetry",
"budget",
"model-outcome",
- "trend-analysis"
+ "trend-analysis",
+ "ci"
]
}
diff --git a/plugins/ruflo-cost-tracker/README.md b/plugins/ruflo-cost-tracker/README.md
index fdf2559ee0..574a5981ee 100644
--- a/plugins/ruflo-cost-tracker/README.md
+++ b/plugins/ruflo-cost-tracker/README.md
@@ -133,7 +133,12 @@ Reserved namespaces (`pattern`, `claude-memories`, `default`) MUST NOT be shadow
```bash
bash plugins/ruflo-cost-tracker/scripts/smoke.sh
-# Expected: "39 passed, 0 failed"
+# Expected: "40 passed, 0 failed"
+
+CI: see [`.github/workflows/cost-tracker-smoke.yml`](../../.github/workflows/cost-tracker-smoke.yml).
+On every PR touching this plugin, GitHub Actions runs smoke + booster-only bench
++ regression gate (Tier 1 winRate β‰₯ 0.80). LLM/Anthropic baselines are NOT run in CI
+β€” they cost real money per invocation and belong in a manual / scheduled workflow.
```
## Architecture Decisions
diff --git a/plugins/ruflo-cost-tracker/scripts/smoke.sh b/plugins/ruflo-cost-tracker/scripts/smoke.sh
index 22710501e9..2e7711e50c 100755
--- a/plugins/ruflo-cost-tracker/scripts/smoke.sh
+++ b/plugins/ruflo-cost-tracker/scripts/smoke.sh
@@ -8,10 +8,10 @@ step() { printf "β†’ %s ... " "$1"; }
ok() { printf "PASS\n"; PASS=$((PASS+1)); }
bad() { printf "FAIL: %s\n" "$1"; FAIL=$((FAIL+1)); }
-step "1. plugin.json declares 0.10.0 with new keywords"
+step "1. plugin.json declares 0.11.0 with new keywords"
v=$(grep -E '"version"' "$ROOT/.claude-plugin/plugin.json" | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1)
-if [[ "$v" != "0.10.0" ]]; then
- bad "expected 0.10.0, got '$v'"
+if [[ "$v" != "0.11.0" ]]; then
+ bad "expected 0.11.0, got '$v'"
else
miss=""
for k in namespace-routing mcp agentic-flow agent-booster tier1-routing model-routing benchmarking verified telemetry budget; do
@@ -321,5 +321,15 @@ step "39. ruflo-cost.md documents 'cost trend' subcommand"
grep -q "cost trend" "$ROOT/commands/ruflo-cost.md" \
&& ok || bad "missing"
+step "40. CI workflow (smoke + booster bench on PR) present"
+WF="$ROOT/../../.github/workflows/cost-tracker-smoke.yml"
+miss=""
+[[ -f "$WF" ]] || miss="$miss missing-file"
+grep -q "smoke\.sh" "$WF" || miss="$miss smoke-not-invoked"
+grep -q "bench\.mjs" "$WF" || miss="$miss bench-not-invoked"
+grep -q "winRate" "$WF" || miss="$miss no-regression-gate"
+grep -q "BENCH_ANTHROPIC\|BENCH_LLM_BASELINE" "$WF" && miss="$miss llm-cost-in-CI"
+[[ -z "$miss" ]] && ok || bad "$miss"
+
printf "\n%s passed, %s failed\n" "$PASS" "$FAIL"
[[ $FAIL -eq 0 ]] || exit 1