Skip to content

Practical patterns, starters & CLI tools for loop engineering with AI coding agents. Design systems that prompt and orchestrate agents (inspired by Addy Osmani and Boris Cherny). Includes loop-audit, loop-init, loop-cost.

fix(pr-babysitter): treat missing checks as unknown (#372)

cyril37d4e09fe44e
diff --git a/patterns/pr-babysitter.md b/patterns/pr-babysitter.md
index 98f671b..e2b040b 100644
--- a/patterns/pr-babysitter.md
+++ b/patterns/pr-babysitter.md
@@ -27,7 +27,11 @@ Keep a small `pr-babysitter-state.md` (or a Linear board / GitHub project view)
Example state entry:
```markdown
- #1234 (feat/auth-refresh)
- Status: Changes requested by @reviewer
+ Checks: passing (unit, lint)
+ Required-check policy: known and satisfied
+ Reviews: changes requested by @reviewer
+ Mergeability: clean
+ Ready to merge: no — changes requested
Last action: Loop proposed minimal diff for comment X
Human decision: Approved the diff, asked for one more test
```
@@ -37,9 +41,15 @@ Example state entry:
1. Discover open PRs authored by the team (or all PRs the user cares about).
2. For each PR:
- Run triage skill.
- - If CI is red → spawn sub-agent with `minimal-fix` skill to address the failure.
+ - Classify checks as passing, failing, pending, or absent/unknown; record
+ whether the repository's required-check policy is known.
+ - If checks are failing → spawn sub-agent with `minimal-fix` skill to address
+ the failure. If pending → wait. If absent/unknown → establish repository
+ policy or escalate instead of assuming green.
- If review comments exist and are actionable → propose minimal patches.
- - If ready (all checks green, approvals present, no blocking comments) → add "ready to merge" label or ping human.
+ - If ready (known policy satisfied, required approvals present, no changes
+ requested or blocking comments, and no merge conflict) → add "ready to
+ merge" label or ping human.
3. For PRs that have been idle too long → suggest close or hand-off.
4. Write concise updates back to the PR and to state.
5. Anything ambiguous or high-risk → surface to human with context.
@@ -93,6 +103,9 @@ PR. See [docs/safety.md](../docs/safety.md).
## Failure Modes & Mitigations
- **Loop proposes bad fixes** → Strong verifier sub-agent (maker/checker) + human review gate for anything beyond trivial.
+- **Missing checks look green** → Represent zero returned checks as
+ `absent/unknown`; a clean/mergeable PR is not ready until repository policy
+ is known and all review/check gates are satisfied.
- **Infinite rebase loops** → Limit number of automated rebase attempts per PR.
- **Stale state** → The loop should prune closed/merged PRs on every run.
- **Notification fatigue** → Use selective notifications (only when human action is truly required).
diff --git a/starters/pr-babysitter-opencode/pr-babysitter-state.md.example b/starters/pr-babysitter-opencode/pr-babysitter-state.md.example
index 9492f41..6a84b45 100644
--- a/starters/pr-babysitter-opencode/pr-babysitter-state.md.example
+++ b/starters/pr-babysitter-opencode/pr-babysitter-state.md.example
@@ -4,7 +4,14 @@ Open PRs tracked: 0
Attempt counts: {}
## Watch List
-- <!-- Add open PRs here -->
+<!-- - #1234 (branch-name)
+ Checks: passing | failing | pending | absent/unknown
+ Required-check policy: known and satisfied | known and unsatisfied | unknown
+ Reviews: approved 1 | changes requested | review required | absent/unknown
+ Mergeability: clean | conflicts | unknown
+ Ready to merge: yes | no — reason
+ Last action: —
+-->
## Fix History (L2+)
- <!-- Record attempt count and outcome per PR -->
diff --git a/starters/pr-babysitter-opencode/skills/pr-review-triage/SKILL.md b/starters/pr-babysitter-opencode/skills/pr-review-triage/SKILL.md
index d8f15b5..d96aa40 100644
--- a/starters/pr-babysitter-opencode/skills/pr-review-triage/SKILL.md
+++ b/starters/pr-babysitter-opencode/skills/pr-review-triage/SKILL.md
@@ -16,18 +16,36 @@ You are a PR babysitter agent. Your job is to track open PRs and surface blocker
- Prior state in `pr-babysitter-state.md`
- CI status for each PR
-## Output
+## Per-PR Output
Update `pr-babysitter-state.md` with:
-- PRs with red CI
-- PRs with merge conflicts
-- PRs waiting >48h for review
-- PRs with unanswered review comments
-- Top 3 actions for a human
+```markdown
+### PR #N — title
+- Checks: passing | failing | pending | absent/unknown — list names and conclusions
+- Required-check policy: known and satisfied | known and unsatisfied | unknown
+- Reviews: approved N | changes requested | review required | absent/unknown
+- Mergeability: clean | conflicts | unknown
+- Blocking comments: (list actionable ones)
+- Ready to merge: yes | no — reason
+- Suggested loop action: none | minimal-fix | rebase | escalate-human
+```
+
+Then list the top 3 actions for a human.
## Rules
+- Zero checks, or no check runs/status contexts returned, means `absent/unknown`,
+ not `passing`, unless the repository policy explicitly requires no checks.
+- Separate functional CI from administrative statuses such as a CLA or labeler;
+ list both, but do not use administrative success as evidence that tests passed.
+- `mergeable` or a clean merge state only means Git found no conflict. It does
+ not mean the PR is ready, reviewed, or verified.
+- "Ready to merge" requires a known project policy, every required check
+ satisfied, required approvals present, no changes requested, no blocking
+ comments, and no merge conflict.
+- If the required-check or review policy cannot be established, report
+ `Ready to merge: no` and escalate to a human.
- Do not edit code in L1 mode.
- Always check for existing PR on the same intent before pushing.
- Security/auth/payments changes: flag for human.
diff --git a/starters/pr-babysitter/.claude/skills/pr-review-triage/SKILL.md b/starters/pr-babysitter/.claude/skills/pr-review-triage/SKILL.md
index 8ea95ff..f399756 100644
--- a/starters/pr-babysitter/.claude/skills/pr-review-triage/SKILL.md
+++ b/starters/pr-babysitter/.claude/skills/pr-review-triage/SKILL.md
@@ -14,8 +14,10 @@ For each watched PR, report:
```markdown
### PR #N — title
-- CI: green | red (job names if red)
-- Reviews: approved N | changes requested | none
+- Checks: passing | failing | pending | absent/unknown — list names and conclusions
+- Required-check policy: known and satisfied | known and unsatisfied | unknown
+- Reviews: approved N | changes requested | review required | absent/unknown
+- Mergeability: clean | conflicts | unknown
- Blocking comments: (list actionable ones)
- Ready to merge: yes | no — reason
- Suggested loop action: none | minimal-fix | rebase | escalate-human
@@ -23,7 +25,17 @@ For each watched PR, report:
## Rules
-- "Ready to merge" requires all required checks + approvals per project policy.
+- Zero checks, or no check runs/status contexts returned, means `absent/unknown`,
+ not `passing`, unless the repository policy explicitly requires no checks.
+- Separate functional CI from administrative statuses such as a CLA or labeler;
+ list both, but do not use administrative success as evidence that tests passed.
+- `mergeable` or a clean merge state only means Git found no conflict. It does
+ not mean the PR is ready, reviewed, or verified.
+- "Ready to merge" requires a known project policy, every required check
+ satisfied, required approvals present, no changes requested, no blocking
+ comments, and no merge conflict.
+- If the required-check or review policy cannot be established, report
+ `Ready to merge: no` and escalate to a human.
- Non-actionable nits → note but do not spawn fix.
- If PR idle >4 days → suggest human handoff.
-- High-risk labels (security, breaking) → escalate-human always.
\ No newline at end of file
+- High-risk labels (security, breaking) → escalate-human always.
diff --git a/starters/pr-babysitter/.codex/skills/pr-review-triage/SKILL.md b/starters/pr-babysitter/.codex/skills/pr-review-triage/SKILL.md
index 8ea95ff..f399756 100644
--- a/starters/pr-babysitter/.codex/skills/pr-review-triage/SKILL.md
+++ b/starters/pr-babysitter/.codex/skills/pr-review-triage/SKILL.md
@@ -14,8 +14,10 @@ For each watched PR, report:
```markdown
### PR #N — title
-- CI: green | red (job names if red)
-- Reviews: approved N | changes requested | none
+- Checks: passing | failing | pending | absent/unknown — list names and conclusions
+- Required-check policy: known and satisfied | known and unsatisfied | unknown
+- Reviews: approved N | changes requested | review required | absent/unknown
+- Mergeability: clean | conflicts | unknown
- Blocking comments: (list actionable ones)
- Ready to merge: yes | no — reason
- Suggested loop action: none | minimal-fix | rebase | escalate-human
@@ -23,7 +25,17 @@ For each watched PR, report:
## Rules
-- "Ready to merge" requires all required checks + approvals per project policy.
+- Zero checks, or no check runs/status contexts returned, means `absent/unknown`,
+ not `passing`, unless the repository policy explicitly requires no checks.
+- Separate functional CI from administrative statuses such as a CLA or labeler;
+ list both, but do not use administrative success as evidence that tests passed.
+- `mergeable` or a clean merge state only means Git found no conflict. It does
+ not mean the PR is ready, reviewed, or verified.
+- "Ready to merge" requires a known project policy, every required check
+ satisfied, required approvals present, no changes requested, no blocking
+ comments, and no merge conflict.
+- If the required-check or review policy cannot be established, report
+ `Ready to merge: no` and escalate to a human.
- Non-actionable nits → note but do not spawn fix.
- If PR idle >4 days → suggest human handoff.
-- High-risk labels (security, breaking) → escalate-human always.
\ No newline at end of file
+- High-risk labels (security, breaking) → escalate-human always.
diff --git a/starters/pr-babysitter/.grok/skills/pr-review-triage/SKILL.md b/starters/pr-babysitter/.grok/skills/pr-review-triage/SKILL.md
index 8ea95ff..f399756 100644
--- a/starters/pr-babysitter/.grok/skills/pr-review-triage/SKILL.md
+++ b/starters/pr-babysitter/.grok/skills/pr-review-triage/SKILL.md
@@ -14,8 +14,10 @@ For each watched PR, report:
```markdown
### PR #N — title
-- CI: green | red (job names if red)
-- Reviews: approved N | changes requested | none
+- Checks: passing | failing | pending | absent/unknown — list names and conclusions
+- Required-check policy: known and satisfied | known and unsatisfied | unknown
+- Reviews: approved N | changes requested | review required | absent/unknown
+- Mergeability: clean | conflicts | unknown
- Blocking comments: (list actionable ones)
- Ready to merge: yes | no — reason
- Suggested loop action: none | minimal-fix | rebase | escalate-human
@@ -23,7 +25,17 @@ For each watched PR, report:
## Rules
-- "Ready to merge" requires all required checks + approvals per project policy.
+- Zero checks, or no check runs/status contexts returned, means `absent/unknown`,
+ not `passing`, unless the repository policy explicitly requires no checks.
+- Separate functional CI from administrative statuses such as a CLA or labeler;
+ list both, but do not use administrative success as evidence that tests passed.
+- `mergeable` or a clean merge state only means Git found no conflict. It does
+ not mean the PR is ready, reviewed, or verified.
+- "Ready to merge" requires a known project policy, every required check
+ satisfied, required approvals present, no changes requested, no blocking
+ comments, and no merge conflict.
+- If the required-check or review policy cannot be established, report
+ `Ready to merge: no` and escalate to a human.
- Non-actionable nits → note but do not spawn fix.
- If PR idle >4 days → suggest human handoff.
-- High-risk labels (security, breaking) → escalate-human always.
\ No newline at end of file
+- High-risk labels (security, breaking) → escalate-human always.
diff --git a/starters/pr-babysitter/pr-babysitter-state.md.example b/starters/pr-babysitter/pr-babysitter-state.md.example
index c133e37..820cdfd 100644
--- a/starters/pr-babysitter/pr-babysitter-state.md.example
+++ b/starters/pr-babysitter/pr-babysitter-state.md.example
@@ -5,7 +5,11 @@ Last run: never
## Watched PRs
<!-- - #1234 (branch-name)
- Status: CI red | changes requested | ready
+ Checks: passing | failing | pending | absent/unknown
+ Required-check policy: known and satisfied | known and unsatisfied | unknown
+ Reviews: approved 1 | changes requested | review required | absent/unknown
+ Mergeability: clean | conflicts | unknown
+ Ready to merge: yes | no — reason
Attempts: 0/3
Last action: —
Human decision: —
@@ -16,4 +20,4 @@ Last run: never
## Resolved (last 7d)
---
-Run log: —
\ No newline at end of file
+Run log: —
diff --git a/tools/loop-init/test/cli.test.mjs b/tools/loop-init/test/cli.test.mjs
index 67a0ad7..616ac64 100644
--- a/tools/loop-init/test/cli.test.mjs
+++ b/tools/loop-init/test/cli.test.mjs
@@ -173,6 +173,28 @@ test('loop-init scaffolds circuit breaker for pr-babysitter (opencode paths)', a
}
});
+test('loop-init scaffolds explicit unknown readiness states for pr-babysitter', async () => {
+ const skillPaths = {
+ grok: ['.grok', 'skills', 'pr-review-triage', 'SKILL.md'],
+ claude: ['.claude', 'skills', 'pr-review-triage', 'SKILL.md'],
+ codex: ['.codex', 'skills', 'pr-review-triage', 'SKILL.md'],
+ opencode: ['skills', 'pr-review-triage', 'SKILL.md'],
+ };
+
+ for (const [tool, skillParts] of Object.entries(skillPaths)) {
+ const dir = await mkdtemp(path.join(tmpdir(), `loop-init-pr-readiness-${tool}-`));
+ try {
+ await exec('node', [CLI, dir, '--pattern', 'pr-babysitter', '--tool', tool]);
+ const skill = await readFile(path.join(dir, ...skillParts), 'utf8');
+ assert.match(skill, /passing \| failing \| pending \| absent\/unknown/);
+ assert.match(skill, /zero checks|no check runs/i);
+ assert.match(skill, /mergeable[\s\S]*does\s+not mean[\s\S]*ready/i);
+ } finally {
+ await rm(dir, { recursive: true, force: true });
+ }
+ }
+});
+
test('loop-init does NOT scaffold circuit breaker for report-only daily-triage', async () => {
const dir = await mkdtemp(path.join(tmpdir(), 'loop-init-nocb-'));
try {