πͺ’ Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
docs(agents): make verification proportionate to risk (#17064)
Nikita Kabardinfc73049ffe44
diff --git a/.agents/AGENTS.md b/.agents/AGENTS.mdindex 58615cc95d..858f262111 100644--- a/.agents/AGENTS.md+++ b/.agents/AGENTS.md@@ -64,12 +64,16 @@ than two sentences they read.- Delegate exploratory or noisy work β broad code search, multi-fileinvestigation, log or test-output trawls β to a subagent so theintermediate tool output stays out of the main context.-- For bug fixes, first write the smallest failing test that proves the reported- behavior and confirm it fails against the buggy behavior before changing- production code. Add another test only when it exercises a distinct adapter,- contract, or execution path. Extend the closest existing test suite; do not- create a standalone constant test when an existing feature suite owns the- behavior. If the bug depends on a data shape, pause and ask: can+- Match verification to risk, not to the fact that you changed something. A test+ earns its place when it pins behavior that could regress without anyone+ noticing. When the only assertion available restates the diff β that a spacing+ value is now that value, that a label reads what it reads β it costs a file and+ proves nothing. Skip it, and say in one line that you did and why.+- When a bug fix does warrant a test, write the smallest failing one first and+ confirm it fails against the buggy behavior before changing production code.+ Add another only when it exercises a distinct adapter, contract, or execution+ path. Extend the closest existing test suite; do not create a standalone+ constant test when an existing feature suite owns the behavior. If the bug depends on a data shape, pause and ask: can`pnpm run seed` prefill that shape locally? If not, consider extending aseeder scenario so the bug stays cheaply reproducible(`packages/shared/scripts/seeder/AGENTS.md`), or note why a seed cannot@@ -273,8 +277,16 @@ A check that passed is not always a check that ran:- `pnpm exec knip` is a required check in `pipeline.yml` with no `package.json`script, so it is easy to never run locally. Unused files and exports under`web/**`, `packages/shared/**` and `worker/**` fail it.-- No check loads a page. A rendering change is not verified until somebody opens- the surface it touches and looks at it.+- No check loads a page, so a rendering change is only verified once somebody+ looks at it β but that somebody need not be you. Drive a browser when you are+ genuinely uncertain: a layout that may reflow, a flow that carries state, an+ interaction whose outcome you cannot predict. When the change is small and+ visual and your confidence is high, say what you changed, hand over the exact+ URL, and let the developer glance at it β that is faster for them than watching+ you automate a confirmation of something you already know. Offering is not+ punting; silently skipping is β so name what you did not check, and offer only+ while somebody is there to take it. If nobody is around and the change is+ user-visible, check it yourself.## Generated Filesdiff --git a/.agents/skills/code-review/SKILL.md b/.agents/skills/code-review/SKILL.mdindex 8b3d3f1f39..1581b818d4 100644--- a/.agents/skills/code-review/SKILL.md+++ b/.agents/skills/code-review/SKILL.md@@ -35,7 +35,8 @@ Focus on:- performance issues with real impact- missing or weak tests for risky changes- Before calling coverage missing, identify the unique regression each proposed- test catches. Prefer merging into the closest existing suite. Flag repeated+ test catches. Do not ask for one whose only assertion would restate the diff β+ a spacing value, a label β because it costs a file and proves nothing. Prefer merging into the closest existing suite. Flag repeatedtests of the same predicate across layers unless each proves a distincttransport, projection, or execution boundary.diff --git a/.agents/skills/frontend-browser-review/SKILL.md b/.agents/skills/frontend-browser-review/SKILL.mdindex 9228eb2ec2..0df9e72e49 100644--- a/.agents/skills/frontend-browser-review/SKILL.md+++ b/.agents/skills/frontend-browser-review/SKILL.md@@ -19,11 +19,26 @@ Use this skill when a change affects what users see or do in the browser.## When To Use It-- UI changes in `web/**`-- Layout, styling, or responsive behavior changes-- Changes to navigation or page flows+Drive a browser when the outcome is uncertain:++- Changes to navigation or page flows, or anything carrying state across steps+- Layout or responsive behavior that could reflow in a way you cannot predict- Bug fixes where the failure mode is visible in the browser-- Final signoff for user-visible frontend work+- Final signoff for a substantial piece of user-visible work++## When To Offer Instead++A small, self-contained visual change you are confident in does not need an+automated pass. Spacing, a colour token, a label, an icon swap: say what you+changed, hand over the exact URL β a local port or the pull request's+`pr-<N>.preview.langfuse.com` β and let the developer take the two-second look.+It is faster for them than waiting while you confirm something you already know,+and a screenshot in the pull request still carries the proof.++Two conditions. **Be explicit** β name what you did not check and why, so the+choice is visible and they can overrule it. And **do it while they are there**:+offering is a handoff, not a way of ending your turn. If nobody is around to+look, and the change is user-visible, check it yourself.## Prefill Test Data Firstdiff --git a/.agents/skills/pr-stack-workflow/SKILL.md b/.agents/skills/pr-stack-workflow/SKILL.mdindex f7d13f0d94..24b2c4992b 100644--- a/.agents/skills/pr-stack-workflow/SKILL.md+++ b/.agents/skills/pr-stack-workflow/SKILL.md@@ -143,7 +143,9 @@ specifically:routinely produce an export whose only consumer lives in a later slice: deadcode on this branch, fine on the tip, and `knip` is a required check. The fixis to move the export into the slice that uses it, not to widen an ignore list.-- **A rendering change is not verified until somebody loads the screen.** Do it- once per slice, on that slice's branch and its own preview, not once on the- tip. Green checks and bot reviews have between them approved a branch that- rendered a blank page.+- **Somebody has to load the screen, per slice.** Green checks and bot reviews+ have between them approved a branch that rendered a blank page, and each slice+ has its own branch and its own preview β so a look at the tip proves nothing+ about slice 3. Who looks is a judgement call `.agents/AGENTS.md` covers: drive+ it yourself where the outcome is uncertain, hand over the preview URL where it+ is not.diff --git a/.agents/skills/refactor-react-effects/SKILL.md b/.agents/skills/refactor-react-effects/SKILL.mdindex 926b0a178a..307758216d 100644--- a/.agents/skills/refactor-react-effects/SKILL.md+++ b/.agents/skills/refactor-react-effects/SKILL.md@@ -145,7 +145,8 @@ Run, at minimum:- ESLint on the migrated path, then `pnpm --filter web run lint`;- type checking when state or action interfaces changed;- a real-browser review for user-visible flows, using seeded data where- applicable;+ applicable β yours where the outcome is uncertain, otherwise the developer's,+ per `.agents/AGENTS.md`;- the effect inventory again to prove the intended scope is clean.Report the exact command summary lines and any retained effects with their