AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents
TypeScript ai-agentai-agent-toolsai-agentsai-agents-frameworkmcpmcp-servermcp-toolsmcpsn8n-alternativeno-code-automationworkflowworkflow-automationworkflows
fix(ci): piece releases no longer fail when npm is still staging a version (#16716)
Hazem Adel8faea8727f62
diff --git a/brain/knowledge/engineering/ci-pipeline.md b/brain/knowledge/engineering/ci-pipeline.mdindex 7d1d2315a09..5f2854090f9 100644--- a/brain/knowledge/engineering/ci-pipeline.md+++ b/brain/knowledge/engineering/ci-pipeline.md@@ -26,7 +26,7 @@ The PR checks that decide whether a change builds and passes tests. Lives in `.g- Two turbo invocations running at the same time in one job race on `cache: false` builds: the retired pipeline failed `api#build` with "@activepieces/shared has no exported member" for a member that exists, because a second invocation was rewriting `shared/dist` while api's tsc read it. One invocation per job.- `--affected` and `--filter` intersect in turbo; there is no union flag. To run "the affected packages plus this fixed set" in one invocation, pass the affected names (the `changes` job outputs them as JSON) as explicit `--filter=<name>` arguments alongside the fixed ones. Two invocations rebuild the shared dependency chain twice because `build` is uncached.- Pieces are the one place the pipeline overrides the dependency graph, by decision (2026-09-24). A piece runs when its own files changed; every piece runs when `pieces/framework` or `pieces/common` changed. A change to `core-piece-types`, `core-utils`, `core-formula` or `core-execution` does not pull the 700+ pieces in, although they depend on it, because that cost about 20 minutes per such PR (5 of the last 60). The accepted blind spot: a core type change that breaks piece compilation is caught when pieces are published (`release-pieces.yml`), not in the PR. The `changes` job computes this with two turbo listings, `ls --filter='[base...HEAD]'` (changed directly) and `ls --affected` (plus dependents).-- Two pushes to main a few minutes apart can both try to publish the same piece version in `release-pieces.yml`. The second run's "already published" check reads npm's `latest` dist-tag, which can stay stale for minutes after a publish, so it publishes again and npm answers `409 Cannot publish over previously staged version` (2026-10-19, `[email protected]`). `publishNpmPackage` treats a failed publish as success when `npm view <name>@<version>` resolves; a `concurrency` group would not help, since the runs did not overlap.+- Two pushes to main a few minutes apart can both try to publish the same piece version in `release-pieces.yml`. The second run's "already published" check reads npm's `latest` dist-tag, which can stay stale for minutes after a publish, so it publishes again and npm answers `409 Cannot publish over previously staged version` (2026-10-19, `[email protected]`). `publishNpmPackage` treats a failed publish as success when `npm view <name>@<version>` resolves; a `concurrency` group would not help, since the runs did not overlap. A staged version can stay invisible to `npm view` for 15+ minutes (`[email protected]`, 2026-10-21, failed two later runs), so on a 409 "previously staged version" `publishNpmPackage` polls `npm view` for up to 5 minutes and passes once the version is live, or fails naming the version and `npm stage list` if it never shows up.- The engine's tests execute five pieces (`AP_DEV_PIECES` in `packages/server/engine/vitest.config.ts`: http, data-mapper, approval, webhook, delay), so those five are engine devDependencies. That edge is what makes `--affected` pull engine in when one of them changes; without it a pieces-only PR ran the piece's checks and skipped the engine tests that exercise it (caught on the retirement PR, 2026-09-27). Three lists name the same set and must move together: that `AP_DEV_PIECES`, `@activepieces/engine#test` `dependsOn` in `turbo.json`, and engine's devDependencies.- `TURBO_SCM_BASE` must be `origin/<base branch>` in CI. turbo's default base is `main`, which a CI checkout does not have, so without it every run degrades to "everything affected".- Required status checks are set by a repo admin (branch ruleset on `main`) and match on job display name. A job its `if:` skips reports `skipped`, which GitHub counts as passing, so every job here can be required: `changes`, `lint`, `build-test`, `api (ce)`, `api (cloud)`, `api (ee, unit, migrations)`, `tool-search (postgres)`. A workflow-level skip (`on.paths`) leaves a required check pending forever, so keep the filtering inside the `changes` job. On 2026-09-27, when the old `ci.yml` was retired, nothing was required yet: `isRequired` was false on every check and the only ruleset enforced codeowner review.diff --git a/tools/scripts/utils/publish-npm-package.ts b/tools/scripts/utils/publish-npm-package.tsindex 01a6df650e9..838d2b3078e 100644--- a/tools/scripts/utils/publish-npm-package.ts+++ b/tools/scripts/utils/publish-npm-package.ts@@ -1,6 +1,7 @@import assert from 'node:assert'import { argv } from 'node:process'-import { execSync } from 'node:child_process'+import { execSync, spawnSync } from 'node:child_process'+import { setTimeout as sleep } from 'node:timers/promises'import { tryCatchSync } from '@activepieces/core-utils'import { existsSync, readFileSync, writeFileSync } from 'node:fs'import { readPackageJson } from './files'@@ -93,10 +94,15 @@ export const publishNpmPackage = async (path: string): Promise<void> => {assertNoUnpublishableDeps(`${outputPath}/package.json`)assertNoSemverRanges(`${outputPath}/package.json`)- const { error } = tryCatchSync(() => execSync(`npm publish --access public --tag latest`, { cwd: outputPath, stdio: 'inherit' }))- if (error) {+ const publish = spawnSync('npm', ['publish', '--access', 'public', '--tag', 'latest'], { cwd: outputPath, stdio: ['ignore', 'inherit', 'pipe'], encoding: 'utf8' })+ process.stderr.write(publish.stderr ?? '')+ if (publish.status !== 0) {+ if (publish.stderr?.includes('previously staged version')) {+ await waitForStagedVersion({ name: json.name, version, path })+ return+ }if (!isVersionOnRegistry({ name: json.name, version })) {- throw error+ throw new Error(`[publishProject] npm publish failed, path=${path}, version=${version}, exitCode=${publish.status}`)}console.info(`[publishProject] publish failed but version is already on registry, path=${path}, version=${version}`)return@@ -110,6 +116,21 @@ function isVersionOnRegistry({ name, version }: { name: string, version: stringreturn data?.trim() === version}+async function waitForStagedVersion({ name, version, path }: { name: string, version: string, path: string }): Promise<void> {+ for (let attempt = 1; attempt <= STAGED_VERSION_POLL_ATTEMPTS; attempt++) {+ console.info(`[publishProject] version staged on registry but not live yet, path=${path}, version=${version}, attempt=${attempt}/${STAGED_VERSION_POLL_ATTEMPTS}`)+ await sleep(STAGED_VERSION_POLL_INTERVAL_MS)+ if (isVersionOnRegistry({ name, version })) {+ console.info(`[publishProject] staged version is now live on registry, path=${path}, version=${version}`)+ return+ }+ }+ throw new Error(`[publishProject] ${name}@${version} is staged on npm but still not live after ${STAGED_VERSION_POLL_ATTEMPTS * STAGED_VERSION_POLL_INTERVAL_MS / 60000} minutes; inspect it with "npm stage list ${name}"`)+}++const STAGED_VERSION_POLL_ATTEMPTS = 20+const STAGED_VERSION_POLL_INTERVAL_MS = 15000+const main = async (): Promise<void> => {const path = argv[2]await publishNpmPackage(path)