Practical patterns, starters & CLI tools for loop engineering with AI coding agents. Design systems that prompt and orchestrate agents (inspired by Addy Osmani and Boris Cherny). Includes loop-audit, loop-init, loop-cost.
TypeScript agentic-aiai-agentsai-codinganthropicautomationclaudeclaude-codecodexcoding-agentsdevops-automationdevtoolsgithub-actionsgrokllmloop-engineeringmcpprompt-engineering
fix(pr-babysitter): treat missing checks as unknown (#372)
cyril37d4e09fe44e
diff --git a/patterns/pr-babysitter.md b/patterns/pr-babysitter.mdindex 98f671b..e2b040b 100644--- a/patterns/pr-babysitter.md+++ b/patterns/pr-babysitter.md@@ -27,7 +27,11 @@ Keep a small `pr-babysitter-state.md` (or a Linear board / GitHub project view)Example state entry:```markdown- #1234 (feat/auth-refresh)- Status: Changes requested by @reviewer+ Checks: passing (unit, lint)+ Required-check policy: known and satisfied+ Reviews: changes requested by @reviewer+ Mergeability: clean+ Ready to merge: no — changes requestedLast action: Loop proposed minimal diff for comment XHuman decision: Approved the diff, asked for one more test```@@ -37,9 +41,15 @@ Example state entry:1. Discover open PRs authored by the team (or all PRs the user cares about).2. For each PR:- Run triage skill.- - If CI is red → spawn sub-agent with `minimal-fix` skill to address the failure.+ - Classify checks as passing, failing, pending, or absent/unknown; record+ whether the repository's required-check policy is known.+ - If checks are failing → spawn sub-agent with `minimal-fix` skill to address+ the failure. If pending → wait. If absent/unknown → establish repository+ policy or escalate instead of assuming green.- If review comments exist and are actionable → propose minimal patches.- - If ready (all checks green, approvals present, no blocking comments) → add "ready to merge" label or ping human.+ - If ready (known policy satisfied, required approvals present, no changes+ requested or blocking comments, and no merge conflict) → add "ready to+ merge" label or ping human.3. For PRs that have been idle too long → suggest close or hand-off.4. Write concise updates back to the PR and to state.5. Anything ambiguous or high-risk → surface to human with context.@@ -93,6 +103,9 @@ PR. See [docs/safety.md](../docs/safety.md).## Failure Modes & Mitigations- **Loop proposes bad fixes** → Strong verifier sub-agent (maker/checker) + human review gate for anything beyond trivial.+- **Missing checks look green** → Represent zero returned checks as+ `absent/unknown`; a clean/mergeable PR is not ready until repository policy+ is known and all review/check gates are satisfied.- **Infinite rebase loops** → Limit number of automated rebase attempts per PR.- **Stale state** → The loop should prune closed/merged PRs on every run.- **Notification fatigue** → Use selective notifications (only when human action is truly required).diff --git a/starters/pr-babysitter-opencode/pr-babysitter-state.md.example b/starters/pr-babysitter-opencode/pr-babysitter-state.md.exampleindex 9492f41..6a84b45 100644--- a/starters/pr-babysitter-opencode/pr-babysitter-state.md.example+++ b/starters/pr-babysitter-opencode/pr-babysitter-state.md.example@@ -4,7 +4,14 @@ Open PRs tracked: 0Attempt counts: {}## Watch List-- <!-- Add open PRs here -->+<!-- - #1234 (branch-name)+ Checks: passing | failing | pending | absent/unknown+ Required-check policy: known and satisfied | known and unsatisfied | unknown+ Reviews: approved 1 | changes requested | review required | absent/unknown+ Mergeability: clean | conflicts | unknown+ Ready to merge: yes | no — reason+ Last action: —+-->## Fix History (L2+)- <!-- Record attempt count and outcome per PR -->diff --git a/starters/pr-babysitter-opencode/skills/pr-review-triage/SKILL.md b/starters/pr-babysitter-opencode/skills/pr-review-triage/SKILL.mdindex d8f15b5..d96aa40 100644--- a/starters/pr-babysitter-opencode/skills/pr-review-triage/SKILL.md+++ b/starters/pr-babysitter-opencode/skills/pr-review-triage/SKILL.md@@ -16,18 +16,36 @@ You are a PR babysitter agent. Your job is to track open PRs and surface blocker- Prior state in `pr-babysitter-state.md`- CI status for each PR-## Output+## Per-PR OutputUpdate `pr-babysitter-state.md` with:-- PRs with red CI-- PRs with merge conflicts-- PRs waiting >48h for review-- PRs with unanswered review comments-- Top 3 actions for a human+```markdown+### PR #N — title+- Checks: passing | failing | pending | absent/unknown — list names and conclusions+- Required-check policy: known and satisfied | known and unsatisfied | unknown+- Reviews: approved N | changes requested | review required | absent/unknown+- Mergeability: clean | conflicts | unknown+- Blocking comments: (list actionable ones)+- Ready to merge: yes | no — reason+- Suggested loop action: none | minimal-fix | rebase | escalate-human+```++Then list the top 3 actions for a human.## Rules+- Zero checks, or no check runs/status contexts returned, means `absent/unknown`,+ not `passing`, unless the repository policy explicitly requires no checks.+- Separate functional CI from administrative statuses such as a CLA or labeler;+ list both, but do not use administrative success as evidence that tests passed.+- `mergeable` or a clean merge state only means Git found no conflict. It does+ not mean the PR is ready, reviewed, or verified.+- "Ready to merge" requires a known project policy, every required check+ satisfied, required approvals present, no changes requested, no blocking+ comments, and no merge conflict.+- If the required-check or review policy cannot be established, report+ `Ready to merge: no` and escalate to a human.- Do not edit code in L1 mode.- Always check for existing PR on the same intent before pushing.- Security/auth/payments changes: flag for human.diff --git a/starters/pr-babysitter/.claude/skills/pr-review-triage/SKILL.md b/starters/pr-babysitter/.claude/skills/pr-review-triage/SKILL.mdindex 8ea95ff..f399756 100644--- a/starters/pr-babysitter/.claude/skills/pr-review-triage/SKILL.md+++ b/starters/pr-babysitter/.claude/skills/pr-review-triage/SKILL.md@@ -14,8 +14,10 @@ For each watched PR, report:```markdown### PR #N — title-- CI: green | red (job names if red)-- Reviews: approved N | changes requested | none+- Checks: passing | failing | pending | absent/unknown — list names and conclusions+- Required-check policy: known and satisfied | known and unsatisfied | unknown+- Reviews: approved N | changes requested | review required | absent/unknown+- Mergeability: clean | conflicts | unknown- Blocking comments: (list actionable ones)- Ready to merge: yes | no — reason- Suggested loop action: none | minimal-fix | rebase | escalate-human@@ -23,7 +25,17 @@ For each watched PR, report:## Rules-- "Ready to merge" requires all required checks + approvals per project policy.+- Zero checks, or no check runs/status contexts returned, means `absent/unknown`,+ not `passing`, unless the repository policy explicitly requires no checks.+- Separate functional CI from administrative statuses such as a CLA or labeler;+ list both, but do not use administrative success as evidence that tests passed.+- `mergeable` or a clean merge state only means Git found no conflict. It does+ not mean the PR is ready, reviewed, or verified.+- "Ready to merge" requires a known project policy, every required check+ satisfied, required approvals present, no changes requested, no blocking+ comments, and no merge conflict.+- If the required-check or review policy cannot be established, report+ `Ready to merge: no` and escalate to a human.- Non-actionable nits → note but do not spawn fix.- If PR idle >4 days → suggest human handoff.-- High-risk labels (security, breaking) → escalate-human always.\ No newline at end of file+- High-risk labels (security, breaking) → escalate-human always.diff --git a/starters/pr-babysitter/.codex/skills/pr-review-triage/SKILL.md b/starters/pr-babysitter/.codex/skills/pr-review-triage/SKILL.mdindex 8ea95ff..f399756 100644--- a/starters/pr-babysitter/.codex/skills/pr-review-triage/SKILL.md+++ b/starters/pr-babysitter/.codex/skills/pr-review-triage/SKILL.md@@ -14,8 +14,10 @@ For each watched PR, report:```markdown### PR #N — title-- CI: green | red (job names if red)-- Reviews: approved N | changes requested | none+- Checks: passing | failing | pending | absent/unknown — list names and conclusions+- Required-check policy: known and satisfied | known and unsatisfied | unknown+- Reviews: approved N | changes requested | review required | absent/unknown+- Mergeability: clean | conflicts | unknown- Blocking comments: (list actionable ones)- Ready to merge: yes | no — reason- Suggested loop action: none | minimal-fix | rebase | escalate-human@@ -23,7 +25,17 @@ For each watched PR, report:## Rules-- "Ready to merge" requires all required checks + approvals per project policy.+- Zero checks, or no check runs/status contexts returned, means `absent/unknown`,+ not `passing`, unless the repository policy explicitly requires no checks.+- Separate functional CI from administrative statuses such as a CLA or labeler;+ list both, but do not use administrative success as evidence that tests passed.+- `mergeable` or a clean merge state only means Git found no conflict. It does+ not mean the PR is ready, reviewed, or verified.+- "Ready to merge" requires a known project policy, every required check+ satisfied, required approvals present, no changes requested, no blocking+ comments, and no merge conflict.+- If the required-check or review policy cannot be established, report+ `Ready to merge: no` and escalate to a human.- Non-actionable nits → note but do not spawn fix.- If PR idle >4 days → suggest human handoff.-- High-risk labels (security, breaking) → escalate-human always.\ No newline at end of file+- High-risk labels (security, breaking) → escalate-human always.diff --git a/starters/pr-babysitter/.grok/skills/pr-review-triage/SKILL.md b/starters/pr-babysitter/.grok/skills/pr-review-triage/SKILL.mdindex 8ea95ff..f399756 100644--- a/starters/pr-babysitter/.grok/skills/pr-review-triage/SKILL.md+++ b/starters/pr-babysitter/.grok/skills/pr-review-triage/SKILL.md@@ -14,8 +14,10 @@ For each watched PR, report:```markdown### PR #N — title-- CI: green | red (job names if red)-- Reviews: approved N | changes requested | none+- Checks: passing | failing | pending | absent/unknown — list names and conclusions+- Required-check policy: known and satisfied | known and unsatisfied | unknown+- Reviews: approved N | changes requested | review required | absent/unknown+- Mergeability: clean | conflicts | unknown- Blocking comments: (list actionable ones)- Ready to merge: yes | no — reason- Suggested loop action: none | minimal-fix | rebase | escalate-human@@ -23,7 +25,17 @@ For each watched PR, report:## Rules-- "Ready to merge" requires all required checks + approvals per project policy.+- Zero checks, or no check runs/status contexts returned, means `absent/unknown`,+ not `passing`, unless the repository policy explicitly requires no checks.+- Separate functional CI from administrative statuses such as a CLA or labeler;+ list both, but do not use administrative success as evidence that tests passed.+- `mergeable` or a clean merge state only means Git found no conflict. It does+ not mean the PR is ready, reviewed, or verified.+- "Ready to merge" requires a known project policy, every required check+ satisfied, required approvals present, no changes requested, no blocking+ comments, and no merge conflict.+- If the required-check or review policy cannot be established, report+ `Ready to merge: no` and escalate to a human.- Non-actionable nits → note but do not spawn fix.- If PR idle >4 days → suggest human handoff.-- High-risk labels (security, breaking) → escalate-human always.\ No newline at end of file+- High-risk labels (security, breaking) → escalate-human always.diff --git a/starters/pr-babysitter/pr-babysitter-state.md.example b/starters/pr-babysitter/pr-babysitter-state.md.exampleindex c133e37..820cdfd 100644--- a/starters/pr-babysitter/pr-babysitter-state.md.example+++ b/starters/pr-babysitter/pr-babysitter-state.md.example@@ -5,7 +5,11 @@ Last run: never## Watched PRs<!-- - #1234 (branch-name)- Status: CI red | changes requested | ready+ Checks: passing | failing | pending | absent/unknown+ Required-check policy: known and satisfied | known and unsatisfied | unknown+ Reviews: approved 1 | changes requested | review required | absent/unknown+ Mergeability: clean | conflicts | unknown+ Ready to merge: yes | no — reasonAttempts: 0/3Last action: —Human decision: —@@ -16,4 +20,4 @@ Last run: never## Resolved (last 7d)----Run log: —\ No newline at end of file+Run log: —diff --git a/tools/loop-init/test/cli.test.mjs b/tools/loop-init/test/cli.test.mjsindex 67a0ad7..616ac64 100644--- a/tools/loop-init/test/cli.test.mjs+++ b/tools/loop-init/test/cli.test.mjs@@ -173,6 +173,28 @@ test('loop-init scaffolds circuit breaker for pr-babysitter (opencode paths)', a}});+test('loop-init scaffolds explicit unknown readiness states for pr-babysitter', async () => {+ const skillPaths = {+ grok: ['.grok', 'skills', 'pr-review-triage', 'SKILL.md'],+ claude: ['.claude', 'skills', 'pr-review-triage', 'SKILL.md'],+ codex: ['.codex', 'skills', 'pr-review-triage', 'SKILL.md'],+ opencode: ['skills', 'pr-review-triage', 'SKILL.md'],+ };++ for (const [tool, skillParts] of Object.entries(skillPaths)) {+ const dir = await mkdtemp(path.join(tmpdir(), `loop-init-pr-readiness-${tool}-`));+ try {+ await exec('node', [CLI, dir, '--pattern', 'pr-babysitter', '--tool', tool]);+ const skill = await readFile(path.join(dir, ...skillParts), 'utf8');+ assert.match(skill, /passing \| failing \| pending \| absent\/unknown/);+ assert.match(skill, /zero checks|no check runs/i);+ assert.match(skill, /mergeable[\s\S]*does\s+not mean[\s\S]*ready/i);+ } finally {+ await rm(dir, { recursive: true, force: true });+ }+ }+});+test('loop-init does NOT scaffold circuit breaker for report-only daily-triage', async () => {const dir = await mkdtemp(path.join(tmpdir(), 'loop-init-nocb-'));try {