πͺ’ Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
docs(agents): a PR-stack skill, the checks that lie, and the handover trigger (#17013)
Nikita Kabardina5784d0d6303
diff --git a/.agents/AGENTS.md b/.agents/AGENTS.mdindex e97c35a964..59f5fbd337 100644--- a/.agents/AGENTS.md+++ b/.agents/AGENTS.md@@ -64,6 +64,30 @@ evaluating, and debugging AI applications.that skip), then resolve it. Do not post `@claude review` again unlessa human asks for another pass.+## Context Handover++Two moments in every task, both easy to skip and both expensive:++- **Before you touch an existing feature, reconstruct its history.** Walk+ commits, the PRs that carried them, and the head branch name β which is where+ the work-item identifier lives β through to the work item itself and any prior+ agent context on it. The commands are in+ `.agents/skills/pr-stack-workflow/references/stack-commands.md` β *Recover the+ context before you slice*. A decision already reversed once does not need+ proposing again.+- **Before you ask for review or merge, leave your reasoning on the work item**+ β the decisions, the reversals, how the human steered, the traps. It survives+ one session otherwise. Do it before the PR, not after the merge: there is no+ later.++The practice, its template and its tooling are the `linear-context-handover` and+`linear-planning` skills in the private `langfuse/langfuse-internal-skills`+plugin, alongside `linear-agent-writes`, which is the policy for what an agent+may write to the tracker and how it must be marked. Read those rather than+improvise. If this environment cannot reach the tracker, say so in your reply and+hand back the text that should have gone on the work item β never skip either+step silently, because silent non-compliance looks exactly like compliance.+## Project Structure```text@@ -165,15 +189,36 @@ langfuse/- Client-bundle soundness: CI scans every prod web build(`pnpm run scan:client-bundle`) for minifier-dropped bindings and Node-onlyglobals leaking into browser chunks β the SWC dropped-binding class ships- runtime-only `ReferenceError`s that dev builds and type checks cannot see- (LFE-10645). On failure, `scripts/scan-client-bundle.mjs`'s header explains- the canonical fix.+ runtime-only `ReferenceError`s that dev builds and type checks cannot see. On+ failure, `scripts/scan-client-bundle.mjs`'s header explains the canonical fix.End your turn with evidence, not claims: quote each check's summary line βe.g. `Tasks: 8 successful, 8 total` (turbo lint/typecheck) or`Tests 12 passed (12)` (vitest) β say which checks you skipped and why,never report unverified work as done, and never end with work pending.+A check that passed is not always a check that ran:++- `lint` and `typecheck` are cached turbo tasks, and worktrees share one cache+ (`using shared worktree cache` on every run), so a pass can be a replay of+ another branch's result. `Tasks: 1 successful, 1 total` prints identically+ either way β quote the `Cached:` line too. To force execution, use+ `pnpm exec turbo run lint --force`; `--no-cache` does not do this, it only+ stops the write (`turbo run lint --help`).+- Every package that lints β `web`, `worker`, `packages/shared`, `ee` β runs+ eslint with `--max-warnings 0`, so one eslint *warning* fails the branch.+- `@langfuse/shared` resolves to its built `dist`. Root `pnpm run typecheck`+ orders that build for you (`turbo.json`: `typecheck.dependsOn` includes+ `^build`), but a filtered `pnpm --filter=web run typecheck` does not β after+ switching a worktree between branches, run+ `pnpm --filter=shared run db:generate && pnpm --filter=shared run build`+ first, or typecheck reports on the previous branch's source.+- `pnpm exec knip` is a required check in `pipeline.yml` with no `package.json`+ script, so it is easy to never run locally. Unused files and exports under+ `web/**`, `packages/shared/**` and `worker/**` fail it.+- No check loads a page. A rendering change is not verified until somebody opens+ the surface it touches and looks at it.+## Generated FilesDo not hand-edit generated or build artifacts:diff --git a/.agents/skills/git-workflow/SKILL.md b/.agents/skills/git-workflow/SKILL.mdindex 61909316c1..4918869f6a 100644--- a/.agents/skills/git-workflow/SKILL.md+++ b/.agents/skills/git-workflow/SKILL.md@@ -53,6 +53,8 @@ operations.- Use `gh search issues` for GitHub issue search.- Prefer non-interactive Git and GitHub commands where possible.- Keep PRs narrow enough to review without unrelated refactors.+- When a change is too large for one reviewable PR, split it into a chained+ stack instead of widening the PR: use `pr-stack-workflow`.- Open PRs as reviewable, not as drafts, unless a human asks for a draft.- Do not post GitHub PR comments as the human author. Cursor agents thatcomment as Cursor should leave one last comment with proof of user-visiblediff --git a/.agents/skills/langfuse-codebase-navigator/SKILL.md b/.agents/skills/langfuse-codebase-navigator/SKILL.mdindex d7305bd999..e356b23bb5 100644--- a/.agents/skills/langfuse-codebase-navigator/SKILL.md+++ b/.agents/skills/langfuse-codebase-navigator/SKILL.md@@ -26,6 +26,7 @@ Use this as the first stop for Langfuse org navigation. Your job is to choose th- Product implementation in `langfuse/langfuse`: choose the matching repo-local skill under `.agents/skills`, such as `backend-dev-guidelines`, `frontend-browser-review`, `clickhouse-best-practices`, `add-model-price`, `turborepo`, `pnpm-upgrade-package`, `code-review`, or production-debug skills as applicable.- Cursor Cloud or Cursor desktop agents implementing a Linear issue, opening a PR, asking a human to test, or handling Claude, Greptile, or Codex review comments: use `cursor-agents-workflow`.+- A change too large for one reviewable PR, splitting a long-lived branch into PRs, or propagating and retargeting a stack: use `pr-stack-workflow`.- Frontend work under `langfuse/langfuse/web`: also check `web/.agents/skills/vercel-react-best-practices` and `web/.agents/skills/vercel-composition-patterns`.- Infrastructure autoscaling or cloud capacity: use `infra-scaling`.- Public Langfuse usage, docs lookup, API access, instrumentation, prompt migration, SDK upgrade, trace analysis, or CLI work: use the public `langfuse` skill from `langfuse/skills`.diff --git a/.agents/skills/pr-stack-workflow/SKILL.md b/.agents/skills/pr-stack-workflow/SKILL.mdnew file mode 100644index 0000000000..f7d13f0d94--- /dev/null+++ b/.agents/skills/pr-stack-workflow/SKILL.md@@ -0,0 +1,149 @@+---+name: pr-stack-workflow+description: |+ Split a large change into a chained stack of small, independently reviewable+ pull requests, and verify each branch on its own. Use when a change is too+ large for one PR, when splitting a long-lived branch into PRs, when deciding+ whether work needs a planning stage at all, when landing or propagating a fix+ through a stack, or when every check is green on a stack and the change is+ still wrong.+---++# PR Stack Workflow++Use this for any change too large to review as one pull request: deciding+whether to stack at all, where to cut the slices, the mechanics that keep a+stack reviewable, and how to verify each branch instead of only the tip.++Branch names, commit and PR-title conventions, and how to handle bot review+threads live in `git-workflow`. Read that first; this skill does not restate it.++## Decide whether it needs a stack++A stack costs real time, because every landing forces a base move plus a merge+down the rest of the chain. Pay that only when one of these is true.++| Stack it | Ship it as one PR |+| --- | --- |+| You cannot describe the diff in one sentence without the word "and" | One symptom, one cause |+| It changes a default, or a shape users already have persisted, so it needs a migration decision | The behavior is new and nobody has state in it |+| Someone will ask "why" and the answer is evidence: numbers, screenshots, a competitor's UI | The bug report is the justification |+| It touches a shared component other surfaces consume | The blast radius is one screen |+| Verification is a click path, not an assertion | An existing test proves it |++Two consequences:++- If the fix is one commit, there is no plan to make. Do not manufacture slices.+- If you cannot yet write the acceptance check, you are researching, not+ planning. Do the research, publish it, then plan.++Keep the stack short. Every landing costs another round of retarget-and-merge,+so length is a running cost you pay until the last PR is in.++## Slice by what a reviewer has to hold in their head++Not by file, not by directory, not by frontend versus backend. A slice is right+when a reviewer can state what it does, and check it, without loading any other+slice. Practical shapes, in the order they should land:++1. **The bugs first.** Anything true on `main` regardless of the wider change.+ It is mergeable on day one, it carries no judgment, and landing it early+ shrinks everything downstream.+2. **Defaults and data shape next.** A default-visible column, a one-time+ migration, "stop rendering an empty string". Cheap diffs with expensive+ consequences deserve their own review.+3. **One screen per slice.** A whole component replaced is one reviewer concern+ even at 30 files.+4. **Housekeeping in a bag.** Several small independent removals nobody will+ argue about.+5. **One question per slice.** "Which run do we compare against?" is one slice:+ the query, the picker, the auto-selection, and the URL state together.+6. **The dense one, alone.** There is always one genuinely hard slice. Give it+ nothing else to carry.+7. **New capabilities last but one.** A new layout or filter reads as a feature+ only once the surface under it has settled.+8. **Instrumentation last.** It wires the seams the earlier slices create. Put+ it first and you instrument code that is about to change.++This repo's own agent-setup work is a worked example: #16390, #16391 and #16392+shipped a project-structure path guard as three PRs, landed in that order β the+measured baseline first, then the rules the guard's messages cite, then the hook+that enforces them. The later two are based on the first's branch, not on `main`,+and #16392's body opens by naming the merge order.++## What each PR must say++State the position and the dependency in the body, so a reviewer knows what they+may ignore:++- `2/8, based on #16913. Base retargets to main when #16913 lands.`+- What is deliberately **not** in this slice, and which slice took it. This is+ the line that stops an implementer helpfully widening the scope and breaking+ the union check below.+- Any unmerged upstream change that would silently break this slice. A change+ that makes two mechanisms read identical data makes them go *quiet* rather+ than visibly wrong, and no test catches that.++Open the whole stack as reviewable, not drafts.++## Five rules that cost us++1. **A fix-up commit lands in the PR that contains the thing it fixes**, never as+ a follow-up PR later in the stack. Otherwise the earlier PR ships a state you+ already know is broken, and it is the one that merges first. When you slice an+ existing branch, fold each fix-up back into its parent commit's slice.+2. **Propagate downward by merging parent into child. Never rebase or force-push+ a stack that is under review.** Review bots hang their threads off commit+ shas, so a force-push discards review you already paid for. These PRs are+ squash-merged, so merge commits in a child cost nothing in `main`'s history.+ (The one documented force-push in `CONTRIBUTING.md` is the CLA author-header+ fix on a single PR β not a stack under review.)+3. **Every PR must stand alone.** Lint, typecheck and tests green on its own+ branch, not only on the stack tip. A slice that compiles only with its+ successor is not a slice: merge it into its neighbor.+4. **Every landing costs a retarget.** When the parent merges, its immediate+ child becomes the new bottom: that one PR's base moves to `main`, `main` is+ merged into it, and the merge is then propagated down the rest of the chain.+ Every landing, until the stack is empty. Plan for that cost instead of+ meeting it on merge day.+5. **A branch that has already merged `main` cannot be split by cherry-picking+ its commits.** The reconciled state exists only in the merged tree: upstream+ took some of your changes, replaced others, reverted a few. Slice the final+ diff instead, then prove the union.++## Prove the union++After slicing a branch, `git diff <stack-tip> <original-branch>` must be empty,+or every remaining difference must be a deliberate, named decision.++Do not skip this because the slices look right. On an eight-PR split the union+delta came to three files and one added, three removed lines β and that delta was+the finding: the original branch had dropped three ESLint disable pragmas and was+carrying a latent lint failure toward `main`.++Keep the original branch untouched as the arbiter. When a slice's behavior is in+doubt, run both and compare. Never edit the arbiter to match a slice.++The commands for slicing a merged branch and proving the union are in+[`references/stack-commands.md`](references/stack-commands.md), together with the+git-to-PR walk for recovering why a surface is the way it is before you slice it.++## Verify per branch, not per stack++Run the verification bar from `.agents/AGENTS.md` on **each branch**, in a+worktree checked out on that branch, and publish the result as a table of branch+by check. That file owns the commands and the checks that pass without executing;+follow it there rather than from memory. Three failure modes belong to stacking+specifically:++- **Worktrees share one turbo cache.** Turbo says so on every run+ (`using shared worktree cache`), which is exactly why a green lint on slice 4+ can be a replay of slice 3's result. Force execution when you switch branches.+- **Slicing is what provokes the unused-export gate.** Split a feature and you+ routinely produce an export whose only consumer lives in a later slice: dead+ code on this branch, fine on the tip, and `knip` is a required check. The fix+ is to move the export into the slice that uses it, not to widen an ignore list.+- **A rendering change is not verified until somebody loads the screen.** Do it+ once per slice, on that slice's branch and its own preview, not once on the+ tip. Green checks and bot reviews have between them approved a branch that+ rendered a blank page.diff --git a/.agents/skills/pr-stack-workflow/agents/openai.yaml b/.agents/skills/pr-stack-workflow/agents/openai.yamlnew file mode 100644index 0000000000..976330f786--- /dev/null+++ b/.agents/skills/pr-stack-workflow/agents/openai.yaml@@ -0,0 +1,4 @@+interface:+ display_name: "PR Stack Workflow"+ short_description: "Slice a large change into chained, reviewable PRs"+ default_prompt: "Use $pr-stack-workflow to decide whether a change needs a PR stack, where to cut the slices, how to propagate and retarget the stack, and how to verify each branch on its own."diff --git a/.agents/skills/pr-stack-workflow/references/stack-commands.md b/.agents/skills/pr-stack-workflow/references/stack-commands.mdnew file mode 100644index 0000000000..02dd6a32d9--- /dev/null+++ b/.agents/skills/pr-stack-workflow/references/stack-commands.md@@ -0,0 +1,130 @@+# Stack commands++Command recipes for the mechanics described in `SKILL.md`. Read this when+building a stack out of an existing branch, propagating a change through one,+retargeting after a landing, or recovering why a surface is the way it is.++- [Recover the context before you slice](#recover-the-context-before-you-slice)+- [Cut a stack from a finished branch](#cut-a-stack-from-a-finished-branch)+- [Prove the union](#prove-the-union)+- [Propagate a change downward](#propagate-a-change-downward)+- [Retarget after a landing](#retarget-after-a-landing)+- [Verify one branch](#verify-one-branch)++## Recover the context before you slice++Most work on an existing surface has a paper trail that answers half the design+questions, and a reversal already litigated once does not need re-litigating.+Walk it from the file to the work item:++```bash+# 1. file -> commits+git log --follow --date=short --format='%h %ad %an %s' -n 10 -- <path>++# 2. commit -> PR: squash-merge subjects carry "(#N)"; otherwise ask GitHub+git log -1 --format=%s <sha>+gh api "repos/langfuse/langfuse/commits/<sha>/pulls" --jq '.[].number'++# 3. PR -> the branch that carries the work-item identifier+gh pr view <N> --json number,state,title,headRefName,baseRefName,url,body++# 4. an identifier -> its other PRs. GitHub search cannot query branch names,+# so match the identifier against recent head branches.+gh pr list --state all --limit 300 --json number,headRefName,title \+ --jq '.[] | select(.headRefName | contains("<identifier>"))'+```++Step 3 is the load-bearing one: this repo keeps work-item identifiers out of+commit messages, PR titles and PR descriptions, so the head branch name is the+only link back to the tracker.++## Cut a stack from a finished branch++A branch that has already merged `main` cannot be split by cherry-picking its+commits β the reconciled state exists only in the merged tree. Slice the final+diff instead, cutting each branch from the previous one:++```bash+git fetch origin+REF=<the finished branch>++# what actually has to land, hunk by hunk+git diff origin/main...$REF --stat+git diff origin/main...$REF -- <paths for slice 1> > /tmp/slice-1.patch++git switch -c stack/01-bugs origin/main+git apply --3way /tmp/slice-1.patch+git add -A && git commit -m "fix: ..."++git switch -c stack/02-defaults stack/01-bugs+# ... repeat, each branch cut from its predecessor+```++Stage with `git add -A`, not `git commit -am`: a slice that adds a file arrives+untracked, and `-a` stages only tracked files, so the committed stack would be+missing it and the union proof below would fail.++Push each branch and open its PR against its parent's branch, not `main`.++## Prove the union++The stack tip must reproduce the finished branch:++```bash+git diff stack/08-instrumentation $REF --stat # expect empty+git diff stack/08-instrumentation $REF # inspect every remaining hunk+```++An empty diff means the split is faithful. A non-empty diff is either a dropped+change or a deliberate decision β name it in the PR body of the slice it belongs+to. Never edit `$REF` to make the diff go away; it is the arbiter.++## Propagate a change downward++A fix belongs in the PR that contains the thing it fixes. Once it is committed+there, merge that branch into each descendant in order. Do not rebase and do not+force-push a branch that has open review threads.++```bash+git switch stack/02-defaults && git merge stack/01-bugs && git push+git switch stack/03-screen && git merge stack/02-defaults && git push+# ... down the stack+```++## Retarget after a landing++When the bottom PR squash-merges, exactly one PR retargets β its immediate+child, which is now the bottom. That child's base moves to `main` and `main` has+to come back in:++```bash+gh pr edit <child PR> --base main+git switch <child branch>+git fetch origin && git merge origin/main && git push+```++The rest of the stack keeps its existing base and picks the landed work up by+the propagate recipe above β `git merge <parent branch>` into each descendant in+order. Retargeting more than one PR to `main` per landing would make every+descendant's diff carry its unmerged ancestors, which is the thing a stack+exists to avoid. This is the running cost of a stack; budget for one round of it+per landing.++## Verify one branch++Per branch, in a worktree checked out on that branch. `.agents/AGENTS.md` owns+the authoritative bar; this is the shape of the loop:++```bash+git worktree add ../langfuse-slice-4 stack/04-housekeeping+cd ../langfuse-slice-4 && pnpm install++pnpm --filter=shared run db:generate && pnpm --filter=shared run build+pnpm exec turbo run lint --force # read the Cached: line, not just Tasks:+pnpm run typecheck+pnpm --filter=web run test-client <file>+pnpm exec knip+```++Then load the surface the slice touches in a browser and look at it. Publish the+outcome as a table of branch by check, and say which checks you skipped.diff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdindex faf89bd2ab..01ade07a5c 100644--- a/CONTRIBUTING.md+++ b/CONTRIBUTING.md@@ -37,6 +37,19 @@ _Before making any significant changes, please [open an issue](https://github.coOnce we've discussed your changes and you've got your code ready, make sure that tests are passing and open your pull request.+Four checks gate every pull request and are cheaper to run before you open it than to discover in CI:++```bash+pnpm run lint # eslint; every package runs with --max-warnings 0, so a warning fails+pnpm tc # typecheck all packages+pnpm exec knip # unused files, exports and dependencies+pnpm run test # see "Running Unit Tests" below for the setup this needs+```++`lint` and `typecheck` are cached, so a pass can be a replay of an earlier run. Read turbo's `Cached:` line as well as its `Tasks:` line, and re-run with `pnpm exec turbo run lint --force` if you need to be sure it executed. For a user-visible change, also open the affected screen in a browser and check it β every pull request gets a full preview deployment at `pr-<N>.preview.langfuse.com`.++If a change is too large to review in one pull request, split it into a chained stack of small PRs rather than widening one. `.agents/skills/pr-stack-workflow/SKILL.md` describes where to cut the slices and how to land them.+A good first step is to search for open [issues](https://github.com/langfuse/langfuse/issues). Issues are labeled, and some good issues to start with are labeled: [good first issue](https://github.com/langfuse/langfuse/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22).## Project Overview