Skip to content

Practical patterns, starters & CLI tools for loop engineering with AI coding agents. Design systems that prompt and orchestrate agents (inspired by Addy Osmani and Boris Cherny). Includes loop-audit, loop-init, loop-cost.

chore(changelog-drafter): add post-run critique section (#77)

Cobus Greylingb5a69e84d5d0
diff --git a/patterns/changelog-drafter.md b/patterns/changelog-drafter.md
index 95f700a..2f60751 100644
--- a/patterns/changelog-drafter.md
+++ b/patterns/changelog-drafter.md
@@ -53,6 +53,21 @@ The loop should prune entries once a release is tagged/published and the draft i
- Tone and project voice
6. On approval: the loop can open a PR that updates `CHANGELOG.md` (or the GitHub release body), or simply leave the draft in a file for the maintainer to copy.
7. Record the run + mark items as "published" in state. Prune old entries.
+8. Record post-run critique in state: missed items, false positives, grouping issues, retries, and prompt/policy adjustments for next run.
+
+## Post-Run Critique
+
+After the human reviews the draft, the human reviewer or operator records what the previous run missed or misclassified so the next run improves. This captures review findings from the last draft run.
+
+Record in state under a `## Post-Run Critique` heading:
+
+- **Missed items** — Changes users reported that the draft omitted
+- **False positives** — Items in the draft that should not have been user-facing (chores, infra-only)
+- **Grouping issues** — Items in the wrong section (e.g., a fix in Features)
+- **Retries** — Number of times the draft was revised or regenerated (+ reason)
+- **Prompt/policy adjustments** — 1–2 concrete changes to scan, draft, or verifier skill instructions for the next run
+
+Optional but recommended during L1 dogfood runs. Even occasional critique entries help prevent repeat issues.
## Verification Strategy
@@ -96,6 +111,7 @@ The loop should prune entries once a release is tagged/published and the draft i
| Overly long / noisy notes | Strict categorization + "user-facing only" rule in the draft skill. Human can trim. |
| Tone mismatch with project | Provide a short "Release voice" section in AGENTS.md or a project skill that the drafter reads. |
| Accidentally publishing | Never grant the loop write access to tags or the live CHANGELOG without an explicit human gate + PR. |
+| Stale critique / never reviewed | Add a human handoff when critique entries accumulate without resolution across a threshold (e.g., 3 runs). |
## Cost Profile
diff --git a/starters/changelog-drafter/changelog-drafter-state.md.example b/starters/changelog-drafter/changelog-drafter-state.md.example
index f7b0390..3249c96 100644
--- a/starters/changelog-drafter/changelog-drafter-state.md.example
+++ b/starters/changelog-drafter/changelog-drafter-state.md.example
@@ -13,6 +13,13 @@ Last release tag: v2.14.0 (2026-06-01)
## Recently Published
- v2.14.0 — published 2026-06-01 (human reviewed draft)
+## Post-Run Critique (from last run)
+- **Missed items**: 0 reported
+- **False positives**: 2 internal chore PRs — tightened bot filter
+- **Grouping issues**: 1 fix in Features — add label check
+- **Retries**: 0
+- **Prompt/policy adjustments**: None needed this run
+
## Scan Window Notes
- Ignore Dependabot / Renovate PRs (handled by dependency-sweeper)
- Direct commits on main are included if they have conventional type or linked issue