πͺ’ Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
docs(agents): add a brake before new concepts in shared backend code (#16792)
Valery Meleshkin08c496a943f7
diff --git a/.agents/skills/backend-dev-guidelines/SKILL.md b/.agents/skills/backend-dev-guidelines/SKILL.mdindex a481b0152d..dae5c6324c 100644--- a/.agents/skills/backend-dev-guidelines/SKILL.md+++ b/.agents/skills/backend-dev-guidelines/SKILL.md@@ -16,6 +16,7 @@ Use this skill for backend and API work across `web/`, `worker/`, and- Building or refactoring backend services and repositories- Working on backend auth, middleware, validation, or observability- Updating Prisma or ClickHouse access patterns+- Adding a field, option, flag, or enum member to shared backend code- Adding or fixing backend tests## How to Read This Skill@@ -29,6 +30,13 @@ Use this skill for backend and API work across `web/`, `worker/`, andshared [`security-review`](../security-review/SKILL.md) skill beforedesigning or implementing the change.+## Before Adding a New Concept++Before adding a field to a shared schema or payload, an option or flag on a+shared signature, an enum member, an env toggle, or a branch that exists for one+caller β or before concluding that no change is needed β read+[`references/new-concepts.md`](references/new-concepts.md).+## Quick Start Checklists### UI: New tRPC Feature@@ -88,6 +96,8 @@ For an additive field/filter:outside env setup.- Validate all external input with Zod v4.- Use Prisma directly for simple CRUD and repositories for complex query access.+- Express new requirements in the vocabulary shared code already has; adding a+ concept to a shared abstraction is the last resort, not the first.- Use OpenTelemetry and DataDog for backend observability.- Always filter project-scoped database queries by `projectId`.- Keep Fern API definitions in sync with public TypeScript API contracts.@@ -129,5 +139,6 @@ For an additive field/filter:| Middleware and auth | You are changing request auth, permissions, or middleware composition | [references/middleware-guide.md](references/middleware-guide.md) || Services and repositories | You are placing business logic, repository code, or DI patterns | [references/services-and-repositories.md](references/services-and-repositories.md) || Database access | You are touching Prisma, ClickHouse, tenant filters, or query patterns | [references/database-patterns.md](references/database-patterns.md) |+| New concepts in shared code | You are adding a field, option, flag, or enum member to a shared abstraction | [references/new-concepts.md](references/new-concepts.md) || Configuration | You are adding env vars, startup config, or runtime toggles | [references/configuration.md](references/configuration.md) || Testing | You are adding or updating backend tests | [references/testing-guide.md](references/testing-guide.md) |diff --git a/.agents/skills/backend-dev-guidelines/references/new-concepts.md b/.agents/skills/backend-dev-guidelines/references/new-concepts.mdnew file mode 100644index 0000000000..ba22a2f387--- /dev/null+++ b/.agents/skills/backend-dev-guidelines/references/new-concepts.md@@ -0,0 +1,201 @@+# Before Adding a New Concept++- [Precedence](#precedence) β these rules yield to an explicit instruction+- [`concept-reformulate-before-extending`](#concept-reformulate-before-extending)+- [`concept-no-exemption-options`](#concept-no-exemption-options)++A new concept is anything that adds vocabulary to shared backend code and that+other people must then learn and maintain:++- a field on a shared schema (Zod contract, declaration schema, queue payload,+ domain type)+- an option or flag on a shared function, service, or repository signature+- a member on a shared enum, status, or state machine+- a branch in shared code that exists for one caller+- an env toggle that changes behavior of code other features depend on+- a new wrapper, layer, or error type placed beside an existing one++Adding one is cheap and looks local: a couple of type-safe lines, tests pass.+Owning one is permanent and global. Every future reader of that abstraction must+learn it, and every future change to it must reason about the interaction between+the new concept and each existing one. In `packages/shared` the multiplier is+worst: `web`, `worker`, and `ee` all consume it, so a concept added for one call+site becomes part of three runtimes' surface.++Among designs whose correctness you can demonstrate, prefer in order:++1. a change confined to your own declaration or call site;+2. a new parameter every caller may set, with one meaning on every path β the+ cheapest form of new vocabulary;+3. a change to shared behavior for all existing callers;+4. any other new concept: one only some callers can use, or that others must+ learn before they can read the abstraction correctly.++Correctness outranks every rung β a tidier declaration that returns wrong+numbers is the expensive option, not the cheap one. Otherwise justification+scales with the rung; 3 and 4 are adjacent in cost, and a design that leans on+an existing declaration inherits its defects.++These two rules are the brake. They are not a prohibition β new concepts are+sometimes correct β they are the check you run first. Both are citable by name,+like the `clickhouse-best-practices` rules: "Per `concept-reformulate-before-extending`β¦".++## Precedence++**These rules are defaults for when the shape of the solution is yours to+choose.** They do not override an explicit instruction. When someone asks for a+particular shape β a named field, a specific mechanism, "make the builder+support this" β say once, briefly, what the cheaper formulation would be and+what the requested one costs, then build what was asked. If they reaffirm, that+is the decision: implement it well and record the tradeoff in the PR, so the+next reader can see the cost was chosen rather than missed.++A requirement that merely *arrives* phrased in mechanism terms is not an+explicit instruction. Inherited framing β a ticket, a handoff, or an earlier+draft that already describes the answer as "a flag thatβ¦" or "a field meaningβ¦"+β is exactly what these rules ask you to re-examine. The precedence above is for+a person choosing a shape on purpose, not for a description that happens to name+one.++---++## `concept-reformulate-before-extending`++**Impact: HIGH** β a new concept multiplies the abstraction's behavior matrix+permanently; a reformulated caller costs nothing.++When a requirement does not fit the vocabulary that already exists, the first+hypothesis is that **the requirement was framed in terms of the first+implementation you reached for**, not that the vocabulary is incomplete. Restate+what the caller needs in the existing vocabulary before concluding a concept is+missing.++Tells that you may be extending prematurely. Each is a signal to check, not a+verdict β weigh them together:++- The name describes a mechanism rather than a domain fact β `preAggregated`,+ `skipNormalization`, `useLegacyPath`, `isSpecialCase`. Naming a mechanism is+ not disqualifying on its own: `explodeArray`, `pairExpand`, and `useFinal` name+ mechanisms the caller genuinely chooses between, and they belong in the+ vocabulary. The distinction is whether the name describes a choice the caller+ is making or the internals of the code that reads it.+- You cannot explain what it means without describing those internals. This is+ the sharper form of the tell above, and it holds on its own.+- Exactly one caller or declaration uses it, and no plausible second one exists.++**Incorrect β a new schema field describing the author's SQL:**++```ts+// "count the expanded rows in my group" is an aggregate at the inner level, so+// neither compilation shape accepts it as written. A new declaration field+// forces one shape and exempts the measure from the other's handling.+toolCallInvocations: {+ sql: "count(*)",+ preAggregated: true, // new field on viewDeclaration, read on one code path+ requiresDimension: "calledToolNames",+}+```++**Correct β the same number, restated so an existing primitive carries it:**++```ts+// "how many times does this tool name occur in the row's array" is a row-level+// expression. The pre-existing @@AGGN@@ template supplies the aggregate the+// inner GROUP BY needs, and the compiler strips it for the other shape.+toolCallInvocations: {+ sql: "countEqual(@@AGG1@@(observations.tool_call_names), calledToolNames)",+ aggs: { agg1: "any" },+ requiresDimension: "calledToolNames",+}+```++Both return per-tool invocation counts. The first needs a schema field plus two+compiler branches; the second needs no change to the shared code at all.++The same move applies outside query code: a queue payload gaining a field the+consumer can already derive from the entity it loads; a service option that+restates a condition the caller could evaluate before calling; a status member+that names a combination of statuses that already exist.++**Procedure.** Before adding the concept, name the existing concepts you+considered and state for each why it cannot express the requirement β in the+plan, then in the PR description. If you cannot write that paragraph, you have+not finished reformulating. If you can, it is the justification a reviewer needs+and the concept is probably right. Concluding the opposite β that an existing+concept already expresses it, so nothing needs adding β is a claim about the+caller's reach, not yours: name the path the requester will actually use, the UI+control or API field and the default they land on. A capability reachable only+by hand-written JSON is missing, not present.++**When one of them comes close but does not work, call that out explicitly** β+name the option and the exact case where it breaks, instead of moving straight+on to a new concept. A near miss is often a defect in that option rather than a+gap in the vocabulary: a grain, nullability, or unit its declaration never+pinned down, which callers have been quietly compensating for. Fixing the+definition can remove the need for both the new concept and the compensation.+If it would change existing callers' results, surface that as its own decision+rather than working around it.++---++## `concept-no-exemption-options`++**Impact: HIGH** β an exemption is untestable in combination and silently scopes+to whichever code path happens to read it.++A concept whose job is to exempt one caller from an invariant is the wrong fix.+The invariant exists because the shared code relies on it; carving one case out+makes it conditional, and every later change must reason about both worlds.++Tells, in order of severity:++1. Its meaning is "do not apply your normal handling to me".+2. It is honored on only one of several code paths, or only when another+ argument has a particular value β so the correct value of one option depends+ on another. That is a hidden contract, and callers who do not care still see+ it in the signature.+3. The follow-up fix adds a *second* rule β a validation error, an allow-list, a+ UI restriction β to contain a hole the first one opened, instead of removing+ the first one.++Tell 3 is decisive. When a fix adds rules rather than removing one, the concept+is wrong, not incomplete.++**Incorrect β a boolean only meaningful for one variant of another option:**++```ts+getObservationsFromEventsTableInternal({+ select: "count" | "rows" | "trace-delete-cursor",+ includeUniqueTraceCount?: boolean, // Only honored with select: "count"+});+```++**Correct β extend the axis that already exists:**++```ts+getObservationsFromEventsTableInternal({+ select: "count" | "count-with-unique-traces" | "rows" | "trace-delete-cursor",+});+```++The same shape appears as a branch that skips shared handling for one+declaration:++```ts+// Incorrect: an escape hatch from the wrap every other declaration receives.+if (metric.requiresDimension && !metric.preAggregated && !sql.includes("(")) {+ sql = `any(${sql})`;+}+```++**Correct:** satisfy the invariant from the caller's side, using the+parameterised primitive the other callers use. If no such primitive exists, add+the *parameter* rather than the exemption β one meaning, available to every+caller, honored on every path.++**Not every alternative to an exemption is cheaper.** Avoiding one by narrowing+shared behavior for every existing caller β removing a choice, tightening a+default, forcing one path β is not automatically the smaller change: it changes+results for callers who never asked. If your change alters what existing callers+get, say which ones and how, and treat it as a decision separate from your+feature.diff --git a/.agents/skills/code-review/references/review-checklist.md b/.agents/skills/code-review/references/review-checklist.mdindex f0ddc0757a..61f5c9aa25 100644--- a/.agents/skills/code-review/references/review-checklist.md+++ b/.agents/skills/code-review/references/review-checklist.md@@ -33,6 +33,10 @@ This is the canonical shared review checklist for Langfuse.- Highlight usage of `redis.call` invocations. Those may have suboptimal redis cluster routing and will raise errors. Instead, use the native call patterns.Example: `await redis?.call("SET", key, "1", "NX", "EX", TTLSeconds);` should use `await redis?.set(key, "1", "EX", TTLSeconds, "NX");` instead.+## New Concepts in Shared Code++- When a change adds vocabulary to shared backend code β a field on a shared schema or payload, an option on a shared signature, an enum member, an env toggle, a branch for one caller β or concludes that no change is needed, review it against [`new-concepts.md`](../../backend-dev-guidelines/references/new-concepts.md) in the [`backend-dev-guidelines`](../../backend-dev-guidelines/SKILL.md) skill.+## Langfuse Cloud- When attempting to confirm if the current environment is Langfuse Cloud in the frontend, use the `useLangfuseCloudRegion` hook and never environment variables directly.diff --git a/packages/shared/AGENTS.md b/packages/shared/AGENTS.mdindex d18d5e3ad5..e1ee201e1f 100644--- a/packages/shared/AGENTS.md+++ b/packages/shared/AGENTS.md@@ -173,6 +173,10 @@ the same PR.- Do not hand-edit generated artifacts under `prisma/generated/*` or `dist/*`.- Avoid exposing server-only modules through `src/index.ts` if they must remainfrontend-safe.+- Adding vocabulary here β a field on a shared schema, an option on a shared+ signature, an enum member, a branch for one caller β is owned by `web`,+ `worker`, and `ee` at once. Apply+ `.agents/skills/backend-dev-guidelines/references/new-concepts.md` first.- Changes to domain constants consumed by blob storage exports (e.g.`LISTABLE_SCORE_TYPES` in `src/domain/scores.ts`, score data type enums)should be reviewed against the blob storage export field reference docs for