AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents
TypeScript ai-agentai-agent-toolsai-agentsai-agents-frameworkmcpmcp-servermcp-toolsmcpsn8n-alternativeno-code-automationworkflowworkflow-automationworkflows
ci(pr-size): give tests their own exempt category (#15674)
Othman Abu Ajamieh85099ba4370e
diff --git a/brain/knowledge/engineering/ci-pr-review-hygiene.md b/brain/knowledge/engineering/ci-pr-review-hygiene.mdindex e8a0e431fc7..bb5df7218db 100644--- a/brain/knowledge/engineering/ci-pr-review-hygiene.md+++ b/brain/knowledge/engineering/ci-pr-review-hygiene.md@@ -10,7 +10,9 @@ The CI gates that shape *how a PR is reviewed*, as opposed to whether it builds.We open PRs as drafts so no human reviewer is auto-assigned until "Ready for review". Greptile's **Review draft pull requests** setting is enabled, so its first pass lands on draft open with no CI glue — first-pass AI review while it is still a draft, human review after. Unlike a once-per-PR CI nudge, the native setting also re-reviews as commits land on the draft.## Per-area size gate-`pr-size.yml` + `tools/scripts/pr-size-check.ts` count **meaningful lines** (additions + deletions, minus lockfiles, `i18n/translation.json`, `locales/**`, snapshots, `dist`) per area and fail when a gated area is over budget: engine+worker+execution combined 300, `core/shared` 250, `server/api` 600, `packages/web` 1200. `packages/pieces` and everything unmatched are measured but exempt — a line count can't tell a cohesive new piece from a codemod, and pieces are self-contained with low blast radius. Bypass with the `large-pr-ok` label or a `revert:` title. Budgets were calibrated from the distribution of recently merged PRs.+`pr-size.yml` + `tools/scripts/pr-size-check.ts` count **meaningful lines** (additions + deletions, minus lockfiles, `i18n/translation.json`, `locales/**`, snapshots, `dist`) per area and fail when a gated area is over budget: engine+worker+execution combined 300, `core/shared` 250, `server/api` 600, `packages/web` 1200. `packages/pieces`, `tests` and everything unmatched are measured but exempt — a line count can't tell a cohesive new piece from a codemod, and pieces are self-contained with low blast radius. Bypass with the `large-pr-ok` label or a `revert:` title. Budgets were calibrated from the distribution of recently merged PRs.++**Tests are their own area and are never charged to the code they cover.** `TEST_PATTERNS` is the first bucket, and first match wins, so `packages/server/api/test/**` lands in `tests`, not `server/api`. It matches a `test`/`tests`/`__tests__`/`__mocks__`/`__snapshots__` path *segment*, a `.test.`/`.spec.` suffix, a `vitest`/`jest`/`playwright`/`checkly` config, `packages/tests-e2e/`, and `smoke-test/`. Otherwise the gate taxes the one thing it should be encouraging: 60% of recently merged PRs carry test lines, and 8 of the last 150 were over budget *only* because of theirs. The budgets were calibrated against diffs that included tests, so this makes them somewhat looser than measured — retune from the new distribution rather than assuming the old numbers still bind the same way.The diff comes from local `git diff --numstat`, not the `/files` API, so it is immune to GitHub's 3,000-file response cap — a mega-PR cannot under-count its way past the gate.@@ -20,6 +22,7 @@ Which team gets asked to review comes entirely from `.github/CODEOWNERS` — theEnforcement is the **`Codeowners review` repository ruleset** (active on the default branch), not classic branch protection: `require_code_owner_review: true` plus `required_approving_review_count: 1` and `required_review_thread_resolution: true`. Eight bypass actors are configured, which is why an owner-team request can look non-blocking on some PRs.## Gotchas+- **The tests bucket keys off a path *segment*, so a feature folder named `test-step` is not mistaken for tests.** `packages/web/src/app/builder/test-step/` and `latest-test.ts` stay in `packages/web`; the regex requires `test/` as a whole segment or a `.test.`/`.spec.` suffix. The one deliberate overlap is `packages/pieces/framework/src/lib/test/index.ts` — shipped `createMockActionContext` helpers, counted as tests, and exempt either way because pieces are exempt.- **A job-level `env:` in `ci.yml` does not reach anything turbo runs.** Turbo 2.x defaults to `--env-mode=strict`: a task's process only receives variables named in `globalPassThroughEnv` / `passThroughEnv` / `env`, plus turbo's own system list. Everything else is dropped silently, so the task falls back to whatever `packages/server/api/.env.tests` set (dotenv there runs without `override`, so a real env var would otherwise win). Symptom: the workflow clearly sets the variable and the test still uses the old value. Add the name to `globalPassThroughEnv` in `turbo.json` — that is why `AP_DB_TYPE` and `AP_POSTGRES_*` are already listed. Jobs that call `npx vitest` directly, like `tool-search-postgres`, are unaffected, which is what makes the difference easy to miss. Verify with a throwaway test that `console.log`s the variable through `npx turbo run <task> --filter=<pkg>`.- **A GitHub service container is not reachable by its name from a non-container job.** `services:` attaches the container to a docker network with a `--network-alias`, but the runner itself is on the host, so the alias does not resolve (`getaddrinfo EAI_AGAIN redis`) and only the published `ports:` mapping on `localhost` works. `.env.tests` pins `AP_REDIS_HOST=redis` for `docker-compose`, so any test that opens a real connection needs `localhost` forced past it.- **Never use `git stash` to prove a new test fails without its fix. Use `git checkout <base> -- <file>` instead.** `git stash push -- <path>` on a path with no uncommitted changes saves nothing and creates no entry, so a following `git stash pop` silently pops whoever's stash is at `stash@{0}` instead. This repo carries long-lived stashes from other branches, so the pop conflicts, is kept, and still writes that stash's untracked files into the working tree, which then look like your own new files. It has happened at least twice, and `stash@{1}` is literally named *"recovered: AGENTS.md agent-skills section (accidentally popped by claude)"*. Reverting one committed file to its base version and running the test there is the same proof with no shared state: `git checkout <merge-base> -- <file>`, run, then `git checkout HEAD -- <file>`. If a stash pop does go wrong, the entry survives the conflict, so the recovery is to delete the stray untracked files after confirming they belong to it with `git stash show --include-untracked --name-only stash@{0}`.diff --git a/tools/scripts/pr-size-check.test.ts b/tools/scripts/pr-size-check.test.tsindex 278c449b572..a8eb1730765 100644--- a/tools/scripts/pr-size-check.test.ts+++ b/tools/scripts/pr-size-check.test.ts@@ -52,6 +52,37 @@ describe('bucketFor', () => {expect(prSizeCheck.bucketFor({ path: 'packages/core/shared/src/index.ts' }).name).toBe('core/shared')})+ it('charges test code to the tests bucket, not the area it covers', () => {+ const areas = [+ 'packages/server/api/test/integration/ce/flows/flow.test.ts',+ 'packages/server/engine/test/handler/loop-executor.test.ts',+ 'packages/core/shared/test/ee/index.test.ts',+ 'packages/web/test/features/chat/lib/messages.test.ts',+ 'packages/web/src/app/builder/data-selector/path-helpers.test.ts',+ 'packages/core/execution/src/lib/step.spec.ts',+ 'packages/pieces/community/simplyprint/src/lib/common/__tests__/client.ts',+ 'packages/server/api/vitest.config.ts',+ 'packages/tests-e2e/scenarios/sign-up.spec.ts',+ 'smoke-test/verify-delay.sh',+ ].map((path) => prSizeCheck.bucketFor({ path }).name)+ expect(new Set(areas)).toEqual(new Set(['tests']))+ })++ it('exempts the tests bucket so adding tests never blocks a PR', () => {+ expect(prSizeCheck.bucketFor({ path: 'packages/server/api/test/integration/ce/flows/flow.test.ts' }).budget).toBeNull()+ })++ it('keeps product code whose path merely contains "test" in its own area', () => {+ const paths = {+ 'packages/web/src/app/builder/test-step/test-trigger-section.tsx': 'packages/web',+ 'packages/server/api/src/app/flow/step-run/step-run.service.ts': 'server/api',+ 'packages/web/src/features/flows/lib/latest-test.ts': 'packages/web',+ }+ for (const [path, area] of Object.entries(paths)) {+ expect(prSizeCheck.bucketFor({ path }).name).toBe(area)+ }+ })+it('exempts pieces and anything unmatched', () => {expect(prSizeCheck.bucketFor({ path: 'packages/pieces/community/slack/src/index.ts' }).budget).toBeNull()expect(prSizeCheck.bucketFor({ path: 'docs/install/overview.mdx' }).name).toBe('other (default)')@@ -92,6 +123,18 @@ describe('collectSizes', () => {expect(report.meaningfulTotal).toBe(10)})+ it('keeps test lines out of the budgeted area totals', () => {+ const numstat = numstatOf([+ [2_000, 0, 'packages/server/api/test/integration/ce/flows/flow.test.ts'],+ [10, 5, 'packages/server/api/src/app/flow/flow.service.ts'],+ ])+ const report = prSizeCheck.collectSizes({ numstat })+ expect(linesOf({ numstat, area: 'server/api' })).toBe(15)+ expect(linesOf({ numstat, area: 'tests' })).toBe(2_000)+ expect(report.rows.some((row) => row.over)).toBe(false)+ expect(report.meaningfulTotal).toBe(2_015)+ })+it('counts renamed files under their destination area', () => {const numstat = numstatOf([[40, 0, 'packages/{shared => core/shared}/src/index.ts']])expect(linesOf({ numstat, area: 'core/shared' })).toBe(40)diff --git a/tools/scripts/pr-size-check.ts b/tools/scripts/pr-size-check.tsindex 21e73206a86..f101d97f2c5 100644--- a/tools/scripts/pr-size-check.ts+++ b/tools/scripts/pr-size-check.ts@@ -11,10 +11,21 @@ const EXCLUDE_PATTERNS: RegExp[] = [/(^|\/)dist\//,]+// Test code, wherever it lives. This bucket is listed first so a test file is charged here+// instead of the product area it covers: a budget that counts tests taxes the PR for adding them.+const TEST_PATTERNS: RegExp[] = [+ /\.(test|spec)\.[cm]?[jt]sx?$/,+ /(^|\/)(test|tests|__tests__|__mocks__|__snapshots__)\//,+ /(^|\/)(vitest|jest|playwright|checkly)\.config\.[cm]?[jt]s$/,+ /^packages\/tests-e2e\//,+ /^smoke-test\//,+]+// First matching bucket wins; `budget: null` means the area is exempt (measured, never gated).// packages/pieces is exempt on purpose: line count can't tell a cohesive new piece from a// codemod or a schema dump, and pieces are @pieces-owned, self-contained, and low blast radius.const BUCKETS: SizeBucket[] = [+ { name: 'tests', tests: TEST_PATTERNS, budget: null },{name: 'engine / worker / execution',tests: [/^packages\/server\/engine\//, /^packages\/server\/worker\//, /^packages\/core\/execution\//],