Practical patterns, starters & CLI tools for loop engineering with AI coding agents. Design systems that prompt and orchestrate agents (inspired by Addy Osmani and Boris Cherny). Includes loop-audit, loop-init, loop-cost.
TypeScript agentic-aiai-agentsai-codinganthropicautomationclaudeclaude-codecodexcoding-agentsdevops-automationdevtoolsgithub-actionsgrokllmloop-engineeringmcpprompt-engineering
feat(loop-guard): size the token budget from loop-cost per pattern (#165)
JonyT927fb7fd936b
diff --git a/templates/SKILL.md.loop-guard b/templates/SKILL.md.loop-guardindex e46a0c6..7e6a9d3 100644--- a/templates/SKILL.md.loop-guard+++ b/templates/SKILL.md.loop-guard@@ -18,12 +18,16 @@ The breaker needs no LLM call, so it is cheap enough to run on every iteration.## The ledger-`loop-ledger.json` records the loop's goal and one entry per attempt:+`loop-ledger.json` records the loop's goal, its pattern/level, and one entry per+attempt:```json-{ "goal": "Get failing CI green", "attempts": [] }+{ "goal": "Get failing CI green", "pattern": "ci-sweeper", "level": "L2", "attempts": [] }```+`pattern` and `level` are seeded by `loop-init`; the breaker ignores them but the+budget step below reads them to size `--token-budget` from real cost data.+After every iteration, append what you just tried:```json@@ -34,12 +38,30 @@ After every iteration, append what you just tried:`outcome` is `success | failure | noop`. Always include `error` on failures —that is how the breaker detects a repeated (stagnant) failure.+## Size the token budget from the pattern (once per run)++Don't hand-type a token cap — derive it from the pattern's realistic per-run+cost so the breaker trips on genuine cost blowup, not a made-up number.+[`loop-cost`](https://github.com/cobusgreyling/loop-engineering/tree/main/tools/loop-cost)+already computes this from `patterns/registry.yaml`; read the loop's `pattern`+and `level` straight from the ledger:++```bash+# Substitute the ledger's own pattern/level (here: ci-sweeper / L2).+BUDGET=$(npx @cobusgreyling/loop-cost --pattern ci-sweeper --level L2 --json \+ | node -e 'let d="";process.stdin.on("data",c=>d+=c).on("end",()=>process.stdout.write(String(JSON.parse(d).scenarios.realistic.tokensPerRun)))')+```++`scenarios.realistic.tokensPerRun` is a rounded integer, so it feeds+`--token-budget` directly. The two tools stay independent — this is shell+wiring, not a code dependency.+## Before each iteration1. Append the previous attempt to `loop-ledger.json`.-2. Run the breaker:+2. Run the breaker with the resolved budget:```bash- npx @cobusgreyling/loop-context --check --ledger loop-ledger.json+ npx @cobusgreyling/loop-context --check --ledger loop-ledger.json --token-budget "$BUDGET"```3. Act on the exit code:- **0** → continue. Optionally trim the next prompt first:@@ -68,4 +90,5 @@ that is how the breaker detects a repeated (stagnant) failure.- `minimal-fix` / `ci-triage` — record each attempt's outcome + error in the ledger.- `loop-verifier` — a verifier rejection is a `failure`; log it so repeats trip the breaker.- `loop-constraints` — honors "escalate after N attempts"; this skill makes it mechanical.-- `loop-budget` — `--token-budget` mirrors the daily cap in loop-budget.md.+- `loop-budget` — the per-run `--token-budget` here comes from loop-cost's+ realistic estimate; loop-budget.md still governs the *daily* cap across runs.diff --git a/tools/loop-cost/README.md b/tools/loop-cost/README.mdindex 7395749..c383cc7 100644--- a/tools/loop-cost/README.md+++ b/tools/loop-cost/README.md@@ -41,4 +41,18 @@ Each estimate includes:Pair with `loop-budget.md` (scaffolded by `loop-init`) and `loop-audit` cost observability checks.+## Feed the circuit breaker++`--json` output resolves a realistic per-run token cap for+[`loop-context`](../loop-context)'s breaker, so `--token-budget` reflects real+pattern cost instead of a hand-typed guess (the `loop-guard` skill wires this):++```bash+BUDGET=$(npx @cobusgreyling/loop-cost --pattern ci-sweeper --level L2 --json \+ | node -e 'let d="";process.stdin.on("data",c=>d+=c).on("end",()=>process.stdout.write(String(JSON.parse(d).scenarios.realistic.tokensPerRun)))')+npx @cobusgreyling/loop-context --check --ledger loop-ledger.json --token-budget "$BUDGET"+```++The tools stay independent — no runtime dependency, just shell wiring.+See [docs/operating-loops.md](../../docs/operating-loops.md).\ No newline at end of filediff --git a/tools/loop-init/README.md b/tools/loop-init/README.mdindex edc9471..38b2f82 100644--- a/tools/loop-init/README.md+++ b/tools/loop-init/README.md@@ -34,9 +34,9 @@ L2 patterns (`ci-sweeper`, `dependency-sweeper`) also copy `minimal-fix` and `loFix-capable patterns (`pr-babysitter`, `ci-sweeper`, `dependency-sweeper`, `post-merge-cleanup`) also get a **circuit breaker**:- `loop-guard` skill — logs each attempt to `loop-ledger.json` and runs [`loop-context`](../loop-context) `--check` before retrying-- `loop-ledger.json` — seeded with the pattern's goal and an empty `attempts` array+- `loop-ledger.json` — seeded with the pattern's goal, its `pattern`/`level`, and an empty `attempts` array-The breaker escalates (same error N× in a row, too many consecutive failures, token budget, or iteration cap) instead of looping in vain. Report-only patterns skip it.+The ledger's `pattern`/`level` let `loop-guard` size the breaker's `--token-budget` from [`loop-cost`](../loop-cost)'s realistic per-run estimate instead of a hand-typed number. The breaker escalates (same error N× in a row, too many consecutive failures, token budget, or iteration cap) instead of looping in vain. Report-only patterns skip it.Every scaffold also creates:diff --git a/tools/loop-init/dist/cli.js b/tools/loop-init/dist/cli.jsindex 0883f2d..3f95e71 100644--- a/tools/loop-init/dist/cli.js+++ b/tools/loop-init/dist/cli.js@@ -152,6 +152,22 @@ const LEDGER_GOAL = {'changelog-drafter': 'Draft accurate release notes','issue-triage': 'Triage the open issue queue',};+/**+ * Readiness level seeded into loop-ledger.json so the loop-guard skill can+ * resolve a realistic per-run token budget from `loop-cost --json` instead of a+ * hand-typed number. Fix-capable loops draft changes with a verifier (a human+ * still merges), so L2 is the right default; tune it in the ledger if a loop+ * runs unattended (L3) or report-only (L1).+ */+const LEDGER_LEVEL = {+ 'daily-triage': 'L1',+ 'pr-babysitter': 'L2',+ 'ci-sweeper': 'L2',+ 'dependency-sweeper': 'L2',+ 'post-merge-cleanup': 'L2',+ 'changelog-drafter': 'L1',+ 'issue-triage': 'L1',+};/*** Fix-capable loops retry actions, so they need a circuit breaker: scaffold the* loop-guard skill plus a seeded loop-ledger.json wired to `loop-context`.@@ -165,7 +181,7 @@ async function scaffoldCircuitBreaker(pattern, tool, targetDir, templatesRoot, dconst ledgerPath = path.join(targetDir, 'loop-ledger.json');if (await exists(ledgerPath))return;- const seed = `${JSON.stringify({ goal: LEDGER_GOAL[pattern], attempts: [] }, null, 2)}\n`;+ const seed = `${JSON.stringify({ goal: LEDGER_GOAL[pattern], pattern, level: LEDGER_LEVEL[pattern], attempts: [] }, null, 2)}\n`;if (dryRun) {console.log(` would write: ${ledgerPath}`);return;diff --git a/tools/loop-init/src/cli.ts b/tools/loop-init/src/cli.tsindex f01413f..c000e49 100644--- a/tools/loop-init/src/cli.ts+++ b/tools/loop-init/src/cli.ts@@ -195,6 +195,23 @@ const LEDGER_GOAL: Record<Pattern, string> = {'issue-triage': 'Triage the open issue queue',};+/**+ * Readiness level seeded into loop-ledger.json so the loop-guard skill can+ * resolve a realistic per-run token budget from `loop-cost --json` instead of a+ * hand-typed number. Fix-capable loops draft changes with a verifier (a human+ * still merges), so L2 is the right default; tune it in the ledger if a loop+ * runs unattended (L3) or report-only (L1).+ */+const LEDGER_LEVEL: Record<Pattern, string> = {+ 'daily-triage': 'L1',+ 'pr-babysitter': 'L2',+ 'ci-sweeper': 'L2',+ 'dependency-sweeper': 'L2',+ 'post-merge-cleanup': 'L2',+ 'changelog-drafter': 'L1',+ 'issue-triage': 'L1',+};+/*** Fix-capable loops retry actions, so they need a circuit breaker: scaffold the* loop-guard skill plus a seeded loop-ledger.json wired to `loop-context`.@@ -214,7 +231,11 @@ async function scaffoldCircuitBreaker(const ledgerPath = path.join(targetDir, 'loop-ledger.json');if (await exists(ledgerPath)) return;- const seed = `${JSON.stringify({ goal: LEDGER_GOAL[pattern], attempts: [] }, null, 2)}\n`;+ const seed = `${JSON.stringify(+ { goal: LEDGER_GOAL[pattern], pattern, level: LEDGER_LEVEL[pattern], attempts: [] },+ null,+ 2,+ )}\n`;if (dryRun) {console.log(` would write: ${ledgerPath}`);return;diff --git a/tools/loop-init/test/cli.test.mjs b/tools/loop-init/test/cli.test.mjsindex 807776d..e6ed91a 100644--- a/tools/loop-init/test/cli.test.mjs+++ b/tools/loop-init/test/cli.test.mjs@@ -131,6 +131,8 @@ test('loop-init scaffolds circuit breaker (loop-guard + ledger) for fix patternsconst ledger = JSON.parse(await readFile(path.join(dir, 'loop-ledger.json'), 'utf8'));assert.equal(typeof ledger.goal, 'string');assert.ok(ledger.goal.length > 0);+ assert.equal(ledger.pattern, 'ci-sweeper');+ assert.match(ledger.level, /^L[123]$/);assert.deepEqual(ledger.attempts, []);} finally {await rm(dir, { recursive: true, force: true });