Skip to content

πŸͺ’ Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

docs(agents): make verification proportionate to risk (#17064)

Nikita Kabardinfc73049ffe44
diff --git a/.agents/AGENTS.md b/.agents/AGENTS.md
index 58615cc95d..858f262111 100644
--- a/.agents/AGENTS.md
+++ b/.agents/AGENTS.md
@@ -64,12 +64,16 @@ than two sentences they read.
- Delegate exploratory or noisy work β€” broad code search, multi-file
investigation, log or test-output trawls β€” to a subagent so the
intermediate tool output stays out of the main context.
-- For bug fixes, first write the smallest failing test that proves the reported
- behavior and confirm it fails against the buggy behavior before changing
- production code. Add another test only when it exercises a distinct adapter,
- contract, or execution path. Extend the closest existing test suite; do not
- create a standalone constant test when an existing feature suite owns the
- behavior. If the bug depends on a data shape, pause and ask: can
+- Match verification to risk, not to the fact that you changed something. A test
+ earns its place when it pins behavior that could regress without anyone
+ noticing. When the only assertion available restates the diff β€” that a spacing
+ value is now that value, that a label reads what it reads β€” it costs a file and
+ proves nothing. Skip it, and say in one line that you did and why.
+- When a bug fix does warrant a test, write the smallest failing one first and
+ confirm it fails against the buggy behavior before changing production code.
+ Add another only when it exercises a distinct adapter, contract, or execution
+ path. Extend the closest existing test suite; do not create a standalone
+ constant test when an existing feature suite owns the behavior. If the bug depends on a data shape, pause and ask: can
`pnpm run seed` prefill that shape locally? If not, consider extending a
seeder scenario so the bug stays cheaply reproducible
(`packages/shared/scripts/seeder/AGENTS.md`), or note why a seed cannot
@@ -273,8 +277,16 @@ A check that passed is not always a check that ran:
- `pnpm exec knip` is a required check in `pipeline.yml` with no `package.json`
script, so it is easy to never run locally. Unused files and exports under
`web/**`, `packages/shared/**` and `worker/**` fail it.
-- No check loads a page. A rendering change is not verified until somebody opens
- the surface it touches and looks at it.
+- No check loads a page, so a rendering change is only verified once somebody
+ looks at it β€” but that somebody need not be you. Drive a browser when you are
+ genuinely uncertain: a layout that may reflow, a flow that carries state, an
+ interaction whose outcome you cannot predict. When the change is small and
+ visual and your confidence is high, say what you changed, hand over the exact
+ URL, and let the developer glance at it β€” that is faster for them than watching
+ you automate a confirmation of something you already know. Offering is not
+ punting; silently skipping is β€” so name what you did not check, and offer only
+ while somebody is there to take it. If nobody is around and the change is
+ user-visible, check it yourself.
## Generated Files
diff --git a/.agents/skills/code-review/SKILL.md b/.agents/skills/code-review/SKILL.md
index 8b3d3f1f39..1581b818d4 100644
--- a/.agents/skills/code-review/SKILL.md
+++ b/.agents/skills/code-review/SKILL.md
@@ -35,7 +35,8 @@ Focus on:
- performance issues with real impact
- missing or weak tests for risky changes
- Before calling coverage missing, identify the unique regression each proposed
- test catches. Prefer merging into the closest existing suite. Flag repeated
+ test catches. Do not ask for one whose only assertion would restate the diff β€”
+ a spacing value, a label β€” because it costs a file and proves nothing. Prefer merging into the closest existing suite. Flag repeated
tests of the same predicate across layers unless each proves a distinct
transport, projection, or execution boundary.
diff --git a/.agents/skills/frontend-browser-review/SKILL.md b/.agents/skills/frontend-browser-review/SKILL.md
index 9228eb2ec2..0df9e72e49 100644
--- a/.agents/skills/frontend-browser-review/SKILL.md
+++ b/.agents/skills/frontend-browser-review/SKILL.md
@@ -19,11 +19,26 @@ Use this skill when a change affects what users see or do in the browser.
## When To Use It
-- UI changes in `web/**`
-- Layout, styling, or responsive behavior changes
-- Changes to navigation or page flows
+Drive a browser when the outcome is uncertain:
+
+- Changes to navigation or page flows, or anything carrying state across steps
+- Layout or responsive behavior that could reflow in a way you cannot predict
- Bug fixes where the failure mode is visible in the browser
-- Final signoff for user-visible frontend work
+- Final signoff for a substantial piece of user-visible work
+
+## When To Offer Instead
+
+A small, self-contained visual change you are confident in does not need an
+automated pass. Spacing, a colour token, a label, an icon swap: say what you
+changed, hand over the exact URL β€” a local port or the pull request's
+`pr-<N>.preview.langfuse.com` β€” and let the developer take the two-second look.
+It is faster for them than waiting while you confirm something you already know,
+and a screenshot in the pull request still carries the proof.
+
+Two conditions. **Be explicit** β€” name what you did not check and why, so the
+choice is visible and they can overrule it. And **do it while they are there**:
+offering is a handoff, not a way of ending your turn. If nobody is around to
+look, and the change is user-visible, check it yourself.
## Prefill Test Data First
diff --git a/.agents/skills/pr-stack-workflow/SKILL.md b/.agents/skills/pr-stack-workflow/SKILL.md
index f7d13f0d94..24b2c4992b 100644
--- a/.agents/skills/pr-stack-workflow/SKILL.md
+++ b/.agents/skills/pr-stack-workflow/SKILL.md
@@ -143,7 +143,9 @@ specifically:
routinely produce an export whose only consumer lives in a later slice: dead
code on this branch, fine on the tip, and `knip` is a required check. The fix
is to move the export into the slice that uses it, not to widen an ignore list.
-- **A rendering change is not verified until somebody loads the screen.** Do it
- once per slice, on that slice's branch and its own preview, not once on the
- tip. Green checks and bot reviews have between them approved a branch that
- rendered a blank page.
+- **Somebody has to load the screen, per slice.** Green checks and bot reviews
+ have between them approved a branch that rendered a blank page, and each slice
+ has its own branch and its own preview β€” so a look at the tip proves nothing
+ about slice 3. Who looks is a judgement call `.agents/AGENTS.md` covers: drive
+ it yourself where the outcome is uncertain, hand over the preview URL where it
+ is not.
diff --git a/.agents/skills/refactor-react-effects/SKILL.md b/.agents/skills/refactor-react-effects/SKILL.md
index 926b0a178a..307758216d 100644
--- a/.agents/skills/refactor-react-effects/SKILL.md
+++ b/.agents/skills/refactor-react-effects/SKILL.md
@@ -145,7 +145,8 @@ Run, at minimum:
- ESLint on the migrated path, then `pnpm --filter web run lint`;
- type checking when state or action interfaces changed;
- a real-browser review for user-visible flows, using seeded data where
- applicable;
+ applicable β€” yours where the outcome is uncertain, otherwise the developer's,
+ per `.agents/AGENTS.md`;
- the effect inventory again to prove the intended scope is clean.
Report the exact command summary lines and any retained effects with their