Skip to content

Practical patterns, starters & CLI tools for loop engineering with AI coding agents. Design systems that prompt and orchestrate agents (inspired by Addy Osmani and Boris Cherny). Includes loop-audit, loop-init, loop-cost.

feat(loop-guard): size the token budget from loop-cost per pattern (#165)

JonyT927fb7fd936b
diff --git a/templates/SKILL.md.loop-guard b/templates/SKILL.md.loop-guard
index e46a0c6..7e6a9d3 100644
--- a/templates/SKILL.md.loop-guard
+++ b/templates/SKILL.md.loop-guard
@@ -18,12 +18,16 @@ The breaker needs no LLM call, so it is cheap enough to run on every iteration.
## The ledger
-`loop-ledger.json` records the loop's goal and one entry per attempt:
+`loop-ledger.json` records the loop's goal, its pattern/level, and one entry per
+attempt:
```json
-{ "goal": "Get failing CI green", "attempts": [] }
+{ "goal": "Get failing CI green", "pattern": "ci-sweeper", "level": "L2", "attempts": [] }
```
+`pattern` and `level` are seeded by `loop-init`; the breaker ignores them but the
+budget step below reads them to size `--token-budget` from real cost data.
+
After every iteration, append what you just tried:
```json
@@ -34,12 +38,30 @@ After every iteration, append what you just tried:
`outcome` is `success | failure | noop`. Always include `error` on failures —
that is how the breaker detects a repeated (stagnant) failure.
+## Size the token budget from the pattern (once per run)
+
+Don't hand-type a token cap — derive it from the pattern's realistic per-run
+cost so the breaker trips on genuine cost blowup, not a made-up number.
+[`loop-cost`](https://github.com/cobusgreyling/loop-engineering/tree/main/tools/loop-cost)
+already computes this from `patterns/registry.yaml`; read the loop's `pattern`
+and `level` straight from the ledger:
+
+```bash
+# Substitute the ledger's own pattern/level (here: ci-sweeper / L2).
+BUDGET=$(npx @cobusgreyling/loop-cost --pattern ci-sweeper --level L2 --json \
+ | node -e 'let d="";process.stdin.on("data",c=>d+=c).on("end",()=>process.stdout.write(String(JSON.parse(d).scenarios.realistic.tokensPerRun)))')
+```
+
+`scenarios.realistic.tokensPerRun` is a rounded integer, so it feeds
+`--token-budget` directly. The two tools stay independent — this is shell
+wiring, not a code dependency.
+
## Before each iteration
1. Append the previous attempt to `loop-ledger.json`.
-2. Run the breaker:
+2. Run the breaker with the resolved budget:
```bash
- npx @cobusgreyling/loop-context --check --ledger loop-ledger.json
+ npx @cobusgreyling/loop-context --check --ledger loop-ledger.json --token-budget "$BUDGET"
```
3. Act on the exit code:
- **0** → continue. Optionally trim the next prompt first:
@@ -68,4 +90,5 @@ that is how the breaker detects a repeated (stagnant) failure.
- `minimal-fix` / `ci-triage` — record each attempt's outcome + error in the ledger.
- `loop-verifier` — a verifier rejection is a `failure`; log it so repeats trip the breaker.
- `loop-constraints` — honors "escalate after N attempts"; this skill makes it mechanical.
-- `loop-budget` — `--token-budget` mirrors the daily cap in loop-budget.md.
+- `loop-budget` — the per-run `--token-budget` here comes from loop-cost's
+ realistic estimate; loop-budget.md still governs the *daily* cap across runs.
diff --git a/tools/loop-cost/README.md b/tools/loop-cost/README.md
index 7395749..c383cc7 100644
--- a/tools/loop-cost/README.md
+++ b/tools/loop-cost/README.md
@@ -41,4 +41,18 @@ Each estimate includes:
Pair with `loop-budget.md` (scaffolded by `loop-init`) and `loop-audit` cost observability checks.
+## Feed the circuit breaker
+
+`--json` output resolves a realistic per-run token cap for
+[`loop-context`](../loop-context)'s breaker, so `--token-budget` reflects real
+pattern cost instead of a hand-typed guess (the `loop-guard` skill wires this):
+
+```bash
+BUDGET=$(npx @cobusgreyling/loop-cost --pattern ci-sweeper --level L2 --json \
+ | node -e 'let d="";process.stdin.on("data",c=>d+=c).on("end",()=>process.stdout.write(String(JSON.parse(d).scenarios.realistic.tokensPerRun)))')
+npx @cobusgreyling/loop-context --check --ledger loop-ledger.json --token-budget "$BUDGET"
+```
+
+The tools stay independent — no runtime dependency, just shell wiring.
+
See [docs/operating-loops.md](../../docs/operating-loops.md).
\ No newline at end of file
diff --git a/tools/loop-init/README.md b/tools/loop-init/README.md
index edc9471..38b2f82 100644
--- a/tools/loop-init/README.md
+++ b/tools/loop-init/README.md
@@ -34,9 +34,9 @@ L2 patterns (`ci-sweeper`, `dependency-sweeper`) also copy `minimal-fix` and `lo
Fix-capable patterns (`pr-babysitter`, `ci-sweeper`, `dependency-sweeper`, `post-merge-cleanup`) also get a **circuit breaker**:
- `loop-guard` skill — logs each attempt to `loop-ledger.json` and runs [`loop-context`](../loop-context) `--check` before retrying
-- `loop-ledger.json` — seeded with the pattern's goal and an empty `attempts` array
+- `loop-ledger.json` — seeded with the pattern's goal, its `pattern`/`level`, and an empty `attempts` array
-The breaker escalates (same error N× in a row, too many consecutive failures, token budget, or iteration cap) instead of looping in vain. Report-only patterns skip it.
+The ledger's `pattern`/`level` let `loop-guard` size the breaker's `--token-budget` from [`loop-cost`](../loop-cost)'s realistic per-run estimate instead of a hand-typed number. The breaker escalates (same error N× in a row, too many consecutive failures, token budget, or iteration cap) instead of looping in vain. Report-only patterns skip it.
Every scaffold also creates:
diff --git a/tools/loop-init/dist/cli.js b/tools/loop-init/dist/cli.js
index 0883f2d..3f95e71 100644
--- a/tools/loop-init/dist/cli.js
+++ b/tools/loop-init/dist/cli.js
@@ -152,6 +152,22 @@ const LEDGER_GOAL = {
'changelog-drafter': 'Draft accurate release notes',
'issue-triage': 'Triage the open issue queue',
};
+/**
+ * Readiness level seeded into loop-ledger.json so the loop-guard skill can
+ * resolve a realistic per-run token budget from `loop-cost --json` instead of a
+ * hand-typed number. Fix-capable loops draft changes with a verifier (a human
+ * still merges), so L2 is the right default; tune it in the ledger if a loop
+ * runs unattended (L3) or report-only (L1).
+ */
+const LEDGER_LEVEL = {
+ 'daily-triage': 'L1',
+ 'pr-babysitter': 'L2',
+ 'ci-sweeper': 'L2',
+ 'dependency-sweeper': 'L2',
+ 'post-merge-cleanup': 'L2',
+ 'changelog-drafter': 'L1',
+ 'issue-triage': 'L1',
+};
/**
* Fix-capable loops retry actions, so they need a circuit breaker: scaffold the
* loop-guard skill plus a seeded loop-ledger.json wired to `loop-context`.
@@ -165,7 +181,7 @@ async function scaffoldCircuitBreaker(pattern, tool, targetDir, templatesRoot, d
const ledgerPath = path.join(targetDir, 'loop-ledger.json');
if (await exists(ledgerPath))
return;
- const seed = `${JSON.stringify({ goal: LEDGER_GOAL[pattern], attempts: [] }, null, 2)}\n`;
+ const seed = `${JSON.stringify({ goal: LEDGER_GOAL[pattern], pattern, level: LEDGER_LEVEL[pattern], attempts: [] }, null, 2)}\n`;
if (dryRun) {
console.log(` would write: ${ledgerPath}`);
return;
diff --git a/tools/loop-init/src/cli.ts b/tools/loop-init/src/cli.ts
index f01413f..c000e49 100644
--- a/tools/loop-init/src/cli.ts
+++ b/tools/loop-init/src/cli.ts
@@ -195,6 +195,23 @@ const LEDGER_GOAL: Record<Pattern, string> = {
'issue-triage': 'Triage the open issue queue',
};
+/**
+ * Readiness level seeded into loop-ledger.json so the loop-guard skill can
+ * resolve a realistic per-run token budget from `loop-cost --json` instead of a
+ * hand-typed number. Fix-capable loops draft changes with a verifier (a human
+ * still merges), so L2 is the right default; tune it in the ledger if a loop
+ * runs unattended (L3) or report-only (L1).
+ */
+const LEDGER_LEVEL: Record<Pattern, string> = {
+ 'daily-triage': 'L1',
+ 'pr-babysitter': 'L2',
+ 'ci-sweeper': 'L2',
+ 'dependency-sweeper': 'L2',
+ 'post-merge-cleanup': 'L2',
+ 'changelog-drafter': 'L1',
+ 'issue-triage': 'L1',
+};
+
/**
* Fix-capable loops retry actions, so they need a circuit breaker: scaffold the
* loop-guard skill plus a seeded loop-ledger.json wired to `loop-context`.
@@ -214,7 +231,11 @@ async function scaffoldCircuitBreaker(
const ledgerPath = path.join(targetDir, 'loop-ledger.json');
if (await exists(ledgerPath)) return;
- const seed = `${JSON.stringify({ goal: LEDGER_GOAL[pattern], attempts: [] }, null, 2)}\n`;
+ const seed = `${JSON.stringify(
+ { goal: LEDGER_GOAL[pattern], pattern, level: LEDGER_LEVEL[pattern], attempts: [] },
+ null,
+ 2,
+ )}\n`;
if (dryRun) {
console.log(` would write: ${ledgerPath}`);
return;
diff --git a/tools/loop-init/test/cli.test.mjs b/tools/loop-init/test/cli.test.mjs
index 807776d..e6ed91a 100644
--- a/tools/loop-init/test/cli.test.mjs
+++ b/tools/loop-init/test/cli.test.mjs
@@ -131,6 +131,8 @@ test('loop-init scaffolds circuit breaker (loop-guard + ledger) for fix patterns
const ledger = JSON.parse(await readFile(path.join(dir, 'loop-ledger.json'), 'utf8'));
assert.equal(typeof ledger.goal, 'string');
assert.ok(ledger.goal.length > 0);
+ assert.equal(ledger.pattern, 'ci-sweeper');
+ assert.match(ledger.level, /^L[123]$/);
assert.deepEqual(ledger.attempts, []);
} finally {
await rm(dir, { recursive: true, force: true });