π The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated
TypeScript agentic-aiagentic-frameworkagentic-workflowagentsai-agentsai-assistantai-skillsautonomous-agentsclaudecodexdsh-pluginharnessmcp-servermulti-agentmulti-agent-systemsnpmskillsswarmswarm-intelligencetypescript
feat(cost-tracker): P8 β GitHub Actions smoke + booster bench on PR (v0.11.0)
Reuven4b3169df91dc
diff --git a/.github/workflows/cost-tracker-smoke.yml b/.github/workflows/cost-tracker-smoke.ymlnew file mode 100644index 0000000000..dca9145b0d--- /dev/null+++ b/.github/workflows/cost-tracker-smoke.yml@@ -0,0 +1,73 @@+# Smoke + booster-only bench for plugins/ruflo-cost-tracker.+#+# Triggers on changes to the plugin or its corpus. Smoke is fast (~100 ms,+# pure bash + node --check) so it always runs. The booster-only bench runs+# locally β installs `agent-booster` in a sibling temp dir then invokes+# bench.mjs from there so node-resolve picks up the package. The LLM and+# Anthropic baselines are intentionally OMITTED: they cost real money per+# run and require Secret Manager keys; they belong in a manual-trigger or+# scheduled workflow with a budget guard, not on every PR.+name: cost-tracker-smoke++on:+ push:+ branches: [main]+ paths:+ - 'plugins/ruflo-cost-tracker/**'+ - '.github/workflows/cost-tracker-smoke.yml'+ pull_request:+ paths:+ - 'plugins/ruflo-cost-tracker/**'+ - '.github/workflows/cost-tracker-smoke.yml'+ workflow_dispatch:++jobs:+ smoke:+ runs-on: ubuntu-latest+ timeout-minutes: 8+ steps:+ - uses: actions/checkout@v4++ - uses: actions/setup-node@v4+ with:+ node-version: '20'++ - name: Run smoke (39+ structural checks)+ run: bash plugins/ruflo-cost-tracker/scripts/smoke.sh++ - name: Install agent-booster for the bench+ run: |+ mkdir -p .ci-bench+ cd .ci-bench+ npm init -y > /dev/null+ # Pin to the same major as v3/node_modules to keep results comparable.+ npm install --no-audit --no-fund --silent agent-booster@^0.2++ - name: Run booster-only bench (no LLM cost)+ run: |+ cd .ci-bench+ node ../plugins/ruflo-cost-tracker/scripts/bench.mjs++ - name: Trend report (drift across runs in this checkout)+ run: node plugins/ruflo-cost-tracker/scripts/trend.mjs+ # The checkout only contains the runs that were committed β useful+ # as a sanity check that trend.mjs runs cleanly on real data.++ - name: Verify Tier 1 win rate β₯ 0.80 (regression gate)+ run: |+ node -e "+ const d = JSON.parse(require('fs').readFileSync('plugins/ruflo-cost-tracker/docs/benchmarks/runs/latest.json'));+ if (d.summary.winRate < 0.80) {+ console.error('REGRESSION: Tier 1 win rate', d.summary.winRate, '< 0.80');+ process.exit(1);+ }+ console.log('Tier 1 win rate:', (d.summary.winRate * 100).toFixed(1) + '%');+ "++ - name: Upload bench artifact+ if: always()+ uses: actions/upload-artifact@v4+ with:+ name: cost-tracker-bench-result+ path: plugins/ruflo-cost-tracker/docs/benchmarks/runs/latest.json+ retention-days: 30diff --git a/plugins/ruflo-cost-tracker/.claude-plugin/plugin.json b/plugins/ruflo-cost-tracker/.claude-plugin/plugin.jsonindex 8c8094f02f..b2f56e1e76 100644--- a/plugins/ruflo-cost-tracker/.claude-plugin/plugin.json+++ b/plugins/ruflo-cost-tracker/.claude-plugin/plugin.json@@ -1,7 +1,7 @@{"name": "ruflo-cost-tracker","description": "Token usage tracking, model cost attribution per agent, budget alerts, and optimization recommendations β uses memory_* (namespace-routed) for cost-tracking and cost-patterns; pairs with federation budget circuit breaker (ADR-097)",- "version": "0.10.0",+ "version": "0.11.0","author": {"name": "ruvnet","url": "https://github.com/ruvnet"@@ -26,6 +26,7 @@"telemetry","budget","model-outcome",- "trend-analysis"+ "trend-analysis",+ "ci"]}diff --git a/plugins/ruflo-cost-tracker/README.md b/plugins/ruflo-cost-tracker/README.mdindex fdf2559ee0..574a5981ee 100644--- a/plugins/ruflo-cost-tracker/README.md+++ b/plugins/ruflo-cost-tracker/README.md@@ -133,7 +133,12 @@ Reserved namespaces (`pattern`, `claude-memories`, `default`) MUST NOT be shadow```bashbash plugins/ruflo-cost-tracker/scripts/smoke.sh-# Expected: "39 passed, 0 failed"+# Expected: "40 passed, 0 failed"++CI: see [`.github/workflows/cost-tracker-smoke.yml`](../../.github/workflows/cost-tracker-smoke.yml).+On every PR touching this plugin, GitHub Actions runs smoke + booster-only bench++ regression gate (Tier 1 winRate β₯ 0.80). LLM/Anthropic baselines are NOT run in CI+β they cost real money per invocation and belong in a manual / scheduled workflow.```## Architecture Decisionsdiff --git a/plugins/ruflo-cost-tracker/scripts/smoke.sh b/plugins/ruflo-cost-tracker/scripts/smoke.shindex 22710501e9..2e7711e50c 100755--- a/plugins/ruflo-cost-tracker/scripts/smoke.sh+++ b/plugins/ruflo-cost-tracker/scripts/smoke.sh@@ -8,10 +8,10 @@ step() { printf "β %s ... " "$1"; }ok() { printf "PASS\n"; PASS=$((PASS+1)); }bad() { printf "FAIL: %s\n" "$1"; FAIL=$((FAIL+1)); }-step "1. plugin.json declares 0.10.0 with new keywords"+step "1. plugin.json declares 0.11.0 with new keywords"v=$(grep -E '"version"' "$ROOT/.claude-plugin/plugin.json" | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1)-if [[ "$v" != "0.10.0" ]]; then- bad "expected 0.10.0, got '$v'"+if [[ "$v" != "0.11.0" ]]; then+ bad "expected 0.11.0, got '$v'"elsemiss=""for k in namespace-routing mcp agentic-flow agent-booster tier1-routing model-routing benchmarking verified telemetry budget; do@@ -321,5 +321,15 @@ step "39. ruflo-cost.md documents 'cost trend' subcommand"grep -q "cost trend" "$ROOT/commands/ruflo-cost.md" \&& ok || bad "missing"+step "40. CI workflow (smoke + booster bench on PR) present"+WF="$ROOT/../../.github/workflows/cost-tracker-smoke.yml"+miss=""+[[ -f "$WF" ]] || miss="$miss missing-file"+grep -q "smoke\.sh" "$WF" || miss="$miss smoke-not-invoked"+grep -q "bench\.mjs" "$WF" || miss="$miss bench-not-invoked"+grep -q "winRate" "$WF" || miss="$miss no-regression-gate"+grep -q "BENCH_ANTHROPIC\|BENCH_LLM_BASELINE" "$WF" && miss="$miss llm-cost-in-CI"+[[ -z "$miss" ]] && ok || bad "$miss"+printf "\n%s passed, %s failed\n" "$PASS" "$FAIL"[[ $FAIL -eq 0 ]] || exit 1