Skip to content

AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents

fix(worker): installing pieces no longer depends on the worker reaching the public URL (#16037)

Ahmad Tashdbc140a31af1
diff --git a/brain/knowledge/decisions/000013-sandbox-pool-is-a-pure-execute-cloud-run-pool-superseded.md b/brain/knowledge/decisions/000013-sandbox-pool-is-a-pure-execute-cloud-run-pool-superseded.md
index f48ecfaaec3..1ae6eb21610 100644
--- a/brain/knowledge/decisions/000013-sandbox-pool-is-a-pure-execute-cloud-run-pool-superseded.md
+++ b/brain/knowledge/decisions/000013-sandbox-pool-is-a-pure-execute-cloud-run-pool-superseded.md
@@ -15,5 +15,5 @@ A pure `execute()` gives local and remote an identical signature and needs no ap
## Consequences
The remote Cloud Run / detached-pool hosting was removed — see "Worker is the Sandbox". The pure-execute shape survives (the Sandbox still holds no app connection and consumes only resolved inputs), but two things premised on a remote host are now retired:
-- **ADR 0002 (pieces are links):** the bundle link being built from `publicApiUrl` was justified by "reachable from a detached remote pool." With no remote pool that reason is gone — the "links" decision still stands on its other merits (transport-uniform install, lazy S3 warm, always-a-working-link for self-hosters), but the public-URL basis is now incidental and could move to the internal URL. ADR files 0001/0002 marked superseded (2026-07-16), not deleted.
-- **Worker→app routing:** for a standalone worker (`AP_CONTAINER_TYPE=WORKER`), `AP_FRONTEND_URL` is the base for ALL per-execution engine→app callbacks (`internalApiUrl`: worker/project, populated-flows, connections/props, run-progress, file I/O, piece serverContext.apiUrl). `publicApiUrl` (from the app's `PUBLIC_URL`) is only string injection (webhook URLs, serverContext.publicUrl) plus the cached, provisioning-time bundle download — not the per-request path. So pointing `AP_FRONTEND_URL` at the cluster-internal service keeps every callback off the public endpoint (avoids DNS + TLS + LB hairpin); it's safe because user-facing/public URLs come independently from `PUBLIC_URL`.
+- **ADR 0002 (pieces are links):** the bundle link being built from `publicApiUrl` was justified by "reachable from a detached remote pool." With no remote pool that reason is gone — the "links" decision still stands on its other merits (transport-uniform install, lazy S3 warm, always-a-working-link for self-hosters), but the public-URL basis is now incidental and has since moved to the internal URL (GIT-1718). ADR files 0001/0002 marked superseded (2026-07-16), not deleted.
+- **Worker→app routing:** for a standalone worker (`AP_CONTAINER_TYPE=WORKER`), `AP_FRONTEND_URL` is the base for ALL per-execution engine→app callbacks (`internalApiUrl`: worker/project, populated-flows, connections/props, run-progress, file I/O, piece serverContext.apiUrl, piece bundle downloads). `publicApiUrl` (from the app's `PUBLIC_URL`) is only string injection (webhook URLs, serverContext.publicUrl) — the worker never fetches from it. So pointing `AP_FRONTEND_URL` at the cluster-internal service keeps every callback off the public endpoint (avoids DNS + TLS + LB hairpin); it's safe because user-facing/public URLs come independently from `PUBLIC_URL`.
diff --git a/brain/knowledge/engineering/helm-chart.md b/brain/knowledge/engineering/helm-chart.md
index 3334622846f..7e261ae8759 100644
--- a/brain/knowledge/engineering/helm-chart.md
+++ b/brain/knowledge/engineering/helm-chart.md
@@ -30,7 +30,7 @@ Only two secrets, both `data: {}` with mittwald `secret-generator` annotations t
- **A user-owned key is pinned once it exists.** If `AP_ENCRYPTION_KEY` / `AP_JWT_SECRET` are in the user's secret when the chart renders, their `secretKeyRef` becomes `optional: false`, so a deleted secret stops the pod instead of letting the generated fallback key silently take over.
- **Never render one env name twice to get precedence.** Helm 4 installs with server-side apply, which rejects duplicate `env` keys ("duplicate entries for key"). A lower-precedence default has to come through `envFrom`.
- **Removing a duplicate env name on a Helm 3 upgrade deletes both entries.** The strategic merge patch matches `env` items by name ("hides previous definition ... may be dropped when using apply"), so a var set in both values keys vanishes from the live pod spec. Never change how an already-duplicated name renders; fix duplicates in the user's values instead.
-- **`AP_FRONTEND_URL` must be reachable from inside the pod.** The worker downloads piece bundles through the public URL, so a `localhost` value boots fine and then fails every flow with `fetch failed`.
+- **Up to 0.92.x, `AP_FRONTEND_URL` must be reachable from inside the pod.** Those releases download piece bundles through the public URL, so a `localhost` value boots fine and then fails every flow with `fetch failed`. Later releases download over the worker's own app URL (`127.0.0.1` in the chart's single container), see the bundle-download gotcha on *Workers*.
- **The mittwald subchart's ClusterRole is named after the release.** Two releases with the same name in different namespaces collide, and deleting a namespace without `helm uninstall` leaves it behind. A second release can set `kubernetes-secret-generator.enabled: false`; the first operator watches every namespace.
- **Helm 4 upgrades of a `Rollout` need `--force-conflicts`.** Argo's controller owns the Service `.spec.selector` (it adds `rollouts-pod-template-hash`), so server-side apply conflicts. After a forced upgrade Argo re-adds the hash and traffic keeps routing.
- **The HPA targets whatever `activepieces.workloadType` resolves to.** It used to target a `Deployment` the chart never creates.
diff --git a/brain/knowledge/execution-runtime/workers.md b/brain/knowledge/execution-runtime/workers.md
index 29aff6e2275..2d23b2c2a9e 100644
--- a/brain/knowledge/execution-runtime/workers.md
+++ b/brain/knowledge/execution-runtime/workers.md
@@ -20,7 +20,8 @@ The deep `Resolver`/`Runtime` concurrency and bundle-caching model lives on the
- `concurrency === 1` primes the sandbox to the full container RAM (cgroup-aware) via `primeFullContainerMemory()`.
### Gotchas
-- **`AP_CONTAINER_TYPE=WORKER` moves every engine→app call onto `AP_FRONTEND_URL`.** In `WORKER_AND_APP` the engine reaches the app at `http://127.0.0.1:<port>/api/`; in `WORKER` it uses `AP_FRONTEND_URL + '/api/'` (`getApiUrl` in `worker/src/lib/config/configs.ts`), and the worker's Socket.IO connection follows it. If that URL is the public hostname, file uploads, run-log uploads and the worker socket all cross the ingress/WAF/CDN, so WAF rules and websocket idle timeouts start failing runs that worked the day before. Point separated workers at the in-cluster app service instead. Found on Alan's fleet 2026-09-29, the day after being advised to switch to `WORKER`.
+- **Piece bundles download over the worker's own `AP_FRONTEND_URL` (`internalApiUrl`), never the app's `PUBLIC_URL` — keep it that way.** Through 0.92.x the sandbox built `GET /v1/engine/pieces/bundle` from the `PUBLIC_URL` the app pushes over the socket, so a standalone worker ignored its own `AP_FRONTEND_URL=http://app` for this one hop. A `localhost` public URL (the docs' default for compose and `docker run -p 8080:80`) failed every uncached install with `fetch failed` (GIT-1838). A public URL that reached a *different* instance sharing the JWT secret returned 404, so custom-piece upload failed with `PIECE_BUNDLE_NOT_AVAILABLE` while the `PACKAGE_ARCHIVE` row sat intact in Postgres. Tell-tale: the bundle GET never shows up in the app's own request log. Fixed by GIT-1718.
+- **`AP_CONTAINER_TYPE=WORKER` moves every engine→app call onto `AP_FRONTEND_URL`.** In `WORKER_AND_APP` the engine reaches the app at `http://127.0.0.1:<port>/api/`; in `WORKER` it uses `AP_FRONTEND_URL + '/api/'` (`getApiUrl` in `worker/src/lib/config/configs.ts`), and the worker's Socket.IO connection follows it. If that URL is the public hostname, file uploads, run-log uploads, piece bundle downloads and the worker socket all cross the ingress/WAF/CDN, so WAF rules and websocket idle timeouts start failing runs that worked the day before. Point separated workers at the in-cluster app service instead. Found on Alan's fleet 2026-09-29, the day after being advised to switch to `WORKER`.
- **The piece workspace pins `linker = "isolated"` in its own `bunfig.toml` — do not delete it, the engine's piece resolver depends on the layout bun chooses.** `pieceInstaller` generates a bun workspace at `<cache>/<version>/common` whose members are `pieces/<pieceName>-<pieceVersion>`, and the engine (`piece-loader.ts` → `resolveInstalledPieceEntry`) resolves a piece **only** at `pieces/<alias>/node_modules/<pieceName>`, which just the isolated linker produces. The workspace used to carry no bunfig and inherited whatever ancestor bunfig bun discovered above the cache mount (`/usr/src/app/bunfig.toml`); when bun instead uses the **hoisted** linker it writes the package as a real directory at the workspace root `node_modules/@activepieces/<name>` and leaves the member with no `node_modules` at all, so every run of that piece fails `PieceNotFoundError: Piece not found for package: <name>-<version>` while `bun install` still exits 0. It was silent and self-perpetuating — `markPiecesAsUsed` writes `ready` and `pieceCheckIfAlreadyInstalled` only looks for *a* `node_modules`, so the piece is reinstalled forever and never resolves. Sep 2026 production: 125/125 folders missing `node_modules` were exactly the 125 packages sitting hoisted at the workspace root; 7,400+ `EXECUTE_FLOW` jobs, 15+ platforms, ~60 official pieces, ~300/hr. Two things had to line up — bun 1.4.0 (Aug 31) changed the layout newly-installed members got, and removing the piece-upgrade Redis gate (#15236, deployed 2026-09-03 08:09 UTC) mass-upgraded every platform's flows to newer piece versions so almost every install was suddenly a *first-time* install. Diagnose on a worker with `for d in <cache>/v*/common/pieces/@activepieces/*/; do [ -d "$d/node_modules" ] || basename $d; done`. Fixed by pinning the linker plus a `LATEST_CACHE_VERSION` bump (v14 → v15) so the fleet rebuilds one clean workspace under the pinned layout rather than carrying a mixed one.
- **No PM2 anymore — the container is the supervision unit.** `docker-entrypoint.sh` launches the bootstrap scripts with plain `node --enable-source-maps` (removed the `pm2-runtime` + `/tmp/ecosystem.config.js` machinery). `APP`/`WORKER` `exec` a single node as PID 1; `WORKER_AND_APP` runs both and, if either exits, kills the other and exits non-zero so the orchestrator restarts the whole container. This drops PM2's *in-container* crash/OOM restart: an OOM-kill no longer silently recycles a child every ~4 min behind a `RestartCount: 0` (the 2026-07-26 wedge's supply of retry attempts — see below); the container now dies and is rescheduled instead. The historical incident notes below still describe the old PM2 behavior as it happened.
- **Version gate (rolling-deploy safety)**: dispatch requires an exact release match, enforced both sides via `versionsAreCompatible` (fail-closed — `undefined` or `UNKNOWN_VERSION` `'0.0.0'` is treated incompatible). App withholds jobs from a mismatched worker (`poll` returns null); worker pauses polling 10s. Ordinary mismatch self-heals on convergence; a read failure does not (cached for process life) and pages on-call once at startup via `assertReleaseReadable`.
diff --git a/packages/server/sandbox/src/lib/cache/local-execution-cache.ts b/packages/server/sandbox/src/lib/cache/local-execution-cache.ts
index 74fc56ff29c..698cc284275 100644
--- a/packages/server/sandbox/src/lib/cache/local-execution-cache.ts
+++ b/packages/server/sandbox/src/lib/cache/local-execution-cache.ts
@@ -13,7 +13,7 @@ export const localExecutionCache = (log: ApLogger, basePath: string, getSettings
async provision({
pieces,
codeSteps,
- publicApiUrl,
+ internalApiUrl,
engineToken,
bestEffort,
}: ProvisionParams): Promise<void> {
@@ -56,7 +56,7 @@ export const localExecutionCache = (log: ApLogger, basePath: string, getSettings
await pieceInstaller(log, basePath, getSettings).install({
pieces: uniquePieces,
includeFilters: true,
- publicApiUrl,
+ internalApiUrl,
engineToken,
bestEffort,
})
@@ -114,7 +114,7 @@ type InstallCodeStepParams = {
type ProvisionParams = {
pieces: PiecePackage[]
codeSteps: CodeArtifact[]
- publicApiUrl: string
+ internalApiUrl: string
engineToken: string
bestEffort?: boolean
}
diff --git a/packages/server/sandbox/src/lib/cache/pieces/piece-installer.ts b/packages/server/sandbox/src/lib/cache/pieces/piece-installer.ts
index a07bd1d5c6f..5ea78a695f7 100644
--- a/packages/server/sandbox/src/lib/cache/pieces/piece-installer.ts
+++ b/packages/server/sandbox/src/lib/cache/pieces/piece-installer.ts
@@ -15,10 +15,10 @@ const relativePiecePath = (piece: PiecePackage) => join('./', 'pieces', `${piece
const piecePath = (rootWorkspace: string, piece: PiecePackage) => join(rootWorkspace, 'pieces', `${piece.pieceName}-${piece.pieceVersion}`)
export const pieceInstaller = (log: ApLogger, basePath: string, getSettings: () => SandboxSettings) => ({
- async install({ pieces, includeFilters, publicApiUrl, engineToken, bestEffort, force }: InstallParams): Promise<void> {
+ async install({ pieces, includeFilters, internalApiUrl, engineToken, bestEffort, force }: InstallParams): Promise<void> {
const groupedPieces = groupPiecesByPackagePath(pieces, basePath, getSettings)
const installPromises = Object.entries(groupedPieces).map(async ([packagePath, piecesInGroup]) => {
- await installPieces(packagePath, piecesInGroup, includeFilters, log, { publicApiUrl, engineToken }, getSettings, bestEffort ?? false, force ?? false)
+ await installPieces(packagePath, piecesInGroup, includeFilters, log, { internalApiUrl, engineToken }, getSettings, bestEffort ?? false, force ?? false)
})
await Promise.all(installPromises)
},
@@ -305,12 +305,12 @@ async function downloadBundleWithRetry({ rootWorkspace, piece, bundleSource }: D
throw lastError
}
-async function downloadBundleIfNotCached({ rootWorkspace, piece, bundleSource: { publicApiUrl, engineToken } }: DownloadBundleParams): Promise<void> {
+async function downloadBundleIfNotCached({ rootWorkspace, piece, bundleSource: { internalApiUrl, engineToken } }: DownloadBundleParams): Promise<void> {
const bundlePath = bundleTgzPath(rootWorkspace, piece)
if (await fileSystemUtils.fileExists(bundlePath)) {
return
}
- const url = pieceBundleEndpointUrl(publicApiUrl, piece)
+ const url = pieceBundleEndpointUrl(internalApiUrl, piece)
const response = await fetch(url, { headers: { Authorization: `Bearer ${engineToken}` } })
if (!response.ok) {
if (response.status === 404 || response.status === 410) {
@@ -325,8 +325,8 @@ async function downloadBundleIfNotCached({ rootWorkspace, piece, bundleSource: {
await writeFile(bundlePath, Buffer.from(await response.arrayBuffer()))
}
-function pieceBundleEndpointUrl(publicApiUrl: string, piece: PiecePackage): string {
- const base = `${ensureTrailingSlash(publicApiUrl)}v1/engine/pieces/bundle`
+function pieceBundleEndpointUrl(internalApiUrl: string, piece: PiecePackage): string {
+ const base = `${ensureTrailingSlash(internalApiUrl)}v1/engine/pieces/bundle`
if (piece.packageType === PackageType.ARCHIVE) {
return `${base}?archiveId=${encodeURIComponent(piece.archiveId)}`
}
@@ -416,14 +416,14 @@ type PieceEntryFileExistsParams = {
type InstallParams = {
pieces: PiecePackage[]
includeFilters: boolean
- publicApiUrl: string
+ internalApiUrl: string
engineToken: string
bestEffort?: boolean
force?: boolean
}
type BundleSource = {
- publicApiUrl: string
+ internalApiUrl: string
engineToken: string
}
diff --git a/packages/server/sandbox/src/lib/engine-rpc-handlers.ts b/packages/server/sandbox/src/lib/engine-rpc-handlers.ts
index 96627c73c83..9a7d5a3073f 100644
--- a/packages/server/sandbox/src/lib/engine-rpc-handlers.ts
+++ b/packages/server/sandbox/src/lib/engine-rpc-handlers.ts
@@ -36,7 +36,7 @@ export const engineRpcHandlers = ({ log, basePath, getSettings, provision }: Eng
await installer.install({
pieces: piecesToReinstall,
includeFilters: true,
- publicApiUrl: provision.publicApiUrl,
+ internalApiUrl: provision.internalApiUrl,
engineToken: provision.engineToken,
force: true,
})
diff --git a/packages/server/sandbox/src/lib/resolver.ts b/packages/server/sandbox/src/lib/resolver.ts
index 968f2ff914a..fb69ca88fc8 100644
--- a/packages/server/sandbox/src/lib/resolver.ts
+++ b/packages/server/sandbox/src/lib/resolver.ts
@@ -40,7 +40,7 @@ export function createResolver({ apiClient, basePath, getSettings, log }: Create
flowVersionId: flowVersion?.id,
pieces: uniquePieces,
codes,
- publicApiUrl: input.publicApiUrl,
+ internalApiUrl: input.internalApiUrl,
engineToken: input.engineToken,
}
return { kind: 'ready', provision, flowVersion }
diff --git a/packages/server/sandbox/src/lib/sandbox.ts b/packages/server/sandbox/src/lib/sandbox.ts
index 7ed25c9c777..b1e5f1324df 100644
--- a/packages/server/sandbox/src/lib/sandbox.ts
+++ b/packages/server/sandbox/src/lib/sandbox.ts
@@ -47,7 +47,7 @@ export function createSandboxRuntime({ concurrency = 1, basePath, getSettings }:
const { error: provisionError } = await tryCatch(() => localExecutionCache(log, basePath, getSettings).provision({
pieces: provision.pieces,
codeSteps: provision.codes,
- publicApiUrl: provision.publicApiUrl,
+ internalApiUrl: provision.internalApiUrl,
engineToken: provision.engineToken,
}))
if (provisionError) {
@@ -119,8 +119,8 @@ export function createSandboxRuntime({ concurrency = 1, basePath, getSettings }:
busy: info.busy,
}))
},
- async prewarm({ log, apiClient, publicApiUrl, flow }: PreWarmSandboxParams): Promise<void> {
- if (isNil(apiClient) || isNil(publicApiUrl)) {
+ async prewarm({ log, apiClient, internalApiUrl, flow }: PreWarmSandboxParams): Promise<void> {
+ if (isNil(apiClient) || isNil(internalApiUrl)) {
return
}
const startedAt = Date.now()
@@ -146,11 +146,11 @@ export function createSandboxRuntime({ concurrency = 1, basePath, getSettings }:
resolver: createResolver({ apiClient, basePath, getSettings, log }),
flows: prewarmData.flows ?? [],
platformId,
- publicApiUrl,
+ internalApiUrl,
engineToken,
log,
})
- await localExecutionCache(log, basePath, getSettings).provision({ pieces, codeSteps, publicApiUrl, engineToken, bestEffort: true })
+ await localExecutionCache(log, basePath, getSettings).provision({ pieces, codeSteps, internalApiUrl, engineToken, bestEffort: true })
log.info({ pieceCount: pieces.length, codeStepCount: codeSteps.length, durationMs: Date.now() - startedAt }, 'Prewarmed sandbox cache')
})
if (error) {
@@ -184,9 +184,9 @@ async function fetchScopeFile({ apiClient, scopeFileId }: FetchScopeFileParams):
return { pieces: content.pieces, codeSteps: content.codes }
}
-async function resolveFlowsForPrewarm({ resolver, flows, platformId, publicApiUrl, engineToken, log }: ResolveFlowsForPrewarmParams): Promise<ResolvedPrewarmInputs> {
+async function resolveFlowsForPrewarm({ resolver, flows, platformId, internalApiUrl, engineToken, log }: ResolveFlowsForPrewarmParams): Promise<ResolvedPrewarmInputs> {
const resolvedFlows = await Promise.all(flows.map(async (flow) => {
- const { data: resolved, error: flowError } = await tryCatch(() => resolver.resolve({ flow, platformId, publicApiUrl, engineToken }))
+ const { data: resolved, error: flowError } = await tryCatch(() => resolver.resolve({ flow, platformId, internalApiUrl, engineToken }))
if (flowError) {
log.warn({ error: String(flowError), flow: { id: flow.id } }, 'Failed to resolve flow for prewarm')
return null
@@ -217,7 +217,7 @@ type ResolveFlowsForPrewarmParams = {
resolver: Resolver
flows: { id: string, versionId: string, projectId: string }[]
platformId: string
- publicApiUrl: string
+ internalApiUrl: string
engineToken: string
log: ApLogger
}
diff --git a/packages/server/sandbox/src/lib/types.ts b/packages/server/sandbox/src/lib/types.ts
index 3b2be41bdc8..bceac8401f5 100644
--- a/packages/server/sandbox/src/lib/types.ts
+++ b/packages/server/sandbox/src/lib/types.ts
@@ -14,7 +14,7 @@ export type Resolver = {
export type ResolveInput = {
platformId: string
- publicApiUrl: string
+ internalApiUrl: string
engineToken: string
flow?: { id: string, versionId: string, projectId: string }
pieces?: PiecePackage[]
@@ -49,22 +49,22 @@ export type ExecuteParams = {
export type PreWarmSandboxParams = {
log: ApLogger
apiClient?: WorkerToApiContract
- publicApiUrl?: string
+ internalApiUrl?: string
// Warm just this flow (e.g. on publish) instead of the platform's whole active set.
flow?: { id: string, versionId: string, projectId: string }
}
-// The Resolver's output and the pool's input. The pool installs each piece straight from a link: it
-// builds `${publicApiUrl}v1/engine/pieces/bundle?name=&version=&token=` per piece and hands that URL
-// to `bun install`, which follows the endpoint's redirect to npm / signed-S3 (or streams the custom
-// archive). No bytes cross the worker socket and the pool never imports WorkerToApiContract; the link
-// is publicApiUrl-based so it is reachable from a remote pool (Cloud Run). See ADR 0002.
+// The Resolver's output and the pool's input. The pool fetches each piece from
+// `${internalApiUrl}v1/engine/pieces/bundle` (engine token as a Bearer header), following the redirect
+// to npm / signed-S3 (or the streamed custom archive), saves it as a local .tgz, and `bun install`s
+// that file. No bytes cross the worker socket and the pool never imports WorkerToApiContract; the URL
+// is the worker's own app URL (ends in `/api/`), the one it already reaches the app on. See ADR 0002.
export type ProvisionInput = {
platformId: string
flowVersionId?: string
pieces: PiecePackage[]
codes: CodeArtifact[]
- publicApiUrl: string
+ internalApiUrl: string
engineToken: string
}
diff --git a/packages/server/sandbox/test/lib/cache/piece-installer.test.ts b/packages/server/sandbox/test/lib/cache/piece-installer.test.ts
index 0b30dec8110..06a9ae2cd2c 100644
--- a/packages/server/sandbox/test/lib/cache/piece-installer.test.ts
+++ b/packages/server/sandbox/test/lib/cache/piece-installer.test.ts
@@ -81,7 +81,7 @@ const fakeLog = {
} as unknown as ApLogger
// Every piece is installed from its bundle link; the dependency value is the engine bundle endpoint.
-const bundleSource = { publicApiUrl: 'http://localhost:3000/api/', engineToken: 'test-token' }
+const bundleSource = { internalApiUrl: 'http://localhost:3000/api/', engineToken: 'test-token' }
const fakeGetSettings = () => ({
EXECUTION_MODE: 'UNSANDBOXED',
@@ -131,6 +131,21 @@ describe('pieceInstaller', () => {
expect(await pathExists(readyFilePath(piece2))).toBe(true)
})
+ it('cold install downloads every bundle from the internal API URL with the engine token', async () => {
+ const registryPiece = makePiece('@activepieces/piece-cold')
+ const archivePiece = makeArchivePiece('@acme/piece-private')
+ const installer = pieceInstaller(fakeLog, testWorkspace, fakeGetSettings)
+
+ mockInstall.mockResolvedValueOnce({ output: '' })
+
+ await installer.install({ pieces: [registryPiece, archivePiece], includeFilters: true, ...bundleSource })
+
+ const expectedInit = { headers: { Authorization: 'Bearer test-token' } }
+ expect(vi.mocked(fetch)).toHaveBeenCalledTimes(2)
+ expect(vi.mocked(fetch)).toHaveBeenCalledWith('http://localhost:3000/api/v1/engine/pieces/bundle?name=%40activepieces%2Fpiece-cold&version=1.0.0', expectedInit)
+ expect(vi.mocked(fetch)).toHaveBeenCalledWith(`http://localhost:3000/api/v1/engine/pieces/bundle?archiveId=${archivePiece.archiveId}`, expectedInit)
+ })
+
it('batch fails with good and bad piece — good piece marked ready, bad piece rolled back', async () => {
const good = makePiece('@activepieces/piece-good')
const bad = makePiece('@activepieces/piece-bad')
diff --git a/packages/server/sandbox/test/lib/sandbox.test.ts b/packages/server/sandbox/test/lib/sandbox.test.ts
index 6f37221ec31..80d633c1560 100644
--- a/packages/server/sandbox/test/lib/sandbox.test.ts
+++ b/packages/server/sandbox/test/lib/sandbox.test.ts
@@ -69,7 +69,7 @@ function buildExecuteParams(workerIndex: number, expiresAt?: number) {
flowVersionId: 'fv1',
pieces: [],
codes: [],
- publicApiUrl: 'http://localhost/api/',
+ internalApiUrl: 'http://localhost/api/',
engineToken: 't',
},
} as never
diff --git a/packages/server/worker/src/lib/api-notify-service.ts b/packages/server/worker/src/lib/api-notify-service.ts
index ecf29088739..5d92fcc25f4 100644
--- a/packages/server/worker/src/lib/api-notify-service.ts
+++ b/packages/server/worker/src/lib/api-notify-service.ts
@@ -2,14 +2,14 @@ import { Runtime } from '@activepieces/sandbox'
import { ApLogger } from '@activepieces/server-utils'
import { ApiToWorkerContract, WorkerToApiContract } from '@activepieces/shared'
-export function createApiToWorkerHandlers({ getRuntime, apiClient, getPublicApiUrl, log }: CreateApiToWorkerHandlersParams): ApiToWorkerContract {
+export function createApiToWorkerHandlers({ getRuntime, apiClient, getInternalApiUrl, log }: CreateApiToWorkerHandlersParams): ApiToWorkerContract {
return {
flowPublished({ flowId, flowVersionId, projectId }) {
log.info({ flowId, flowVersionId, projectId, message: 'Flow published, prewarming flow cache' })
void getRuntime()?.prewarm({
log,
apiClient,
- publicApiUrl: getPublicApiUrl(),
+ internalApiUrl: getInternalApiUrl(),
flow: { id: flowId, versionId: flowVersionId, projectId },
})
},
@@ -19,6 +19,6 @@ export function createApiToWorkerHandlers({ getRuntime, apiClient, getPublicApiU
type CreateApiToWorkerHandlersParams = {
getRuntime: () => Runtime | null
apiClient: WorkerToApiContract
- getPublicApiUrl: () => string
+ getInternalApiUrl: () => string
log: ApLogger
}
diff --git a/packages/server/worker/src/lib/execute/jobs/execute-action.ts b/packages/server/worker/src/lib/execute/jobs/execute-action.ts
index 2e432905c9f..66a9f5d2697 100644
--- a/packages/server/worker/src/lib/execute/jobs/execute-action.ts
+++ b/packages/server/worker/src/lib/execute/jobs/execute-action.ts
@@ -10,7 +10,7 @@ export const executeActionJob: JobHandler<ExecuteActionJobData, SynchronousJobRe
jobType: WorkerJobType.EXECUTE_ACTION,
async execute(ctx: JobContext, data: ExecuteActionJobData): Promise<SynchronousJobResult> {
const { codes, namespace: codeNamespace } = await resolveCodeStep({ step: data.step, platformId: data.platformId })
- const resolved = await ctx.resolver.resolve({ platformId: data.platformId, publicApiUrl: ctx.publicApiUrl, engineToken: ctx.engineToken, pieces: data.piece ? [data.piece] : [], codes })
+ const resolved = await ctx.resolver.resolve({ platformId: data.platformId, internalApiUrl: ctx.internalApiUrl, engineToken: ctx.engineToken, pieces: data.piece ? [data.piece] : [], codes })
if (resolved.kind !== 'ready') {
throw new Error(`Unexpected resolve outcome "${resolved.kind}" for action-run action job`)
}
diff --git a/packages/server/worker/src/lib/execute/jobs/execute-flow.ts b/packages/server/worker/src/lib/execute/jobs/execute-flow.ts
index 176e7c74b96..b82c98880ad 100644
--- a/packages/server/worker/src/lib/execute/jobs/execute-flow.ts
+++ b/packages/server/worker/src/lib/execute/jobs/execute-flow.ts
@@ -13,7 +13,7 @@ export const executeFlowJob: JobHandler<ExecuteFlowJobData, FireAndForgetJobResu
const timeoutInSeconds = workerSettings.getSettings().FLOW_TIMEOUT_SECONDS
const { data: resolved, error: provisionError } = await tryCatch(() =>
- ctx.resolver.resolve({ platformId: data.platformId, publicApiUrl: ctx.publicApiUrl, engineToken: ctx.engineToken, flow: { id: data.flowId, versionId: data.flowVersionId, projectId: data.projectId } }),
+ ctx.resolver.resolve({ platformId: data.platformId, internalApiUrl: ctx.internalApiUrl, engineToken: ctx.engineToken, flow: { id: data.flowId, versionId: data.flowVersionId, projectId: data.projectId } }),
)
if (provisionError) {
await reportFlowStatus({ ctx, data, status: FlowRunStatus.INTERNAL_ERROR, internalError: toInternalError(RunInternalErrorSource.WORKER, provisionError) })
diff --git a/packages/server/worker/src/lib/execute/jobs/execute-polling.ts b/packages/server/worker/src/lib/execute/jobs/execute-polling.ts
index 80b57d4933e..f7cfe507eb1 100644
--- a/packages/server/worker/src/lib/execute/jobs/execute-polling.ts
+++ b/packages/server/worker/src/lib/execute/jobs/execute-polling.ts
@@ -10,7 +10,7 @@ export const executePollingJob: JobHandler<PollingJobData, FireAndForgetJobResul
async execute(ctx: JobContext, data: PollingJobData): Promise<FireAndForgetJobResult> {
const timeoutInSeconds = workerSettings.getSettings().TRIGGER_TIMEOUT_SECONDS
- const resolved = await ctx.resolver.resolve({ platformId: data.platformId, publicApiUrl: ctx.publicApiUrl, engineToken: ctx.engineToken, flow: { id: data.flowId, versionId: data.flowVersionId, projectId: data.projectId } })
+ const resolved = await ctx.resolver.resolve({ platformId: data.platformId, internalApiUrl: ctx.internalApiUrl, engineToken: ctx.engineToken, flow: { id: data.flowId, versionId: data.flowVersionId, projectId: data.projectId } })
if (resolved.kind === 'flow-not-found') {
ctx.log.info({ flowVersion: { id: data.flowVersionId } }, 'Flow version not found for polling trigger, skipping')
diff --git a/packages/server/worker/src/lib/execute/jobs/execute-property.ts b/packages/server/worker/src/lib/execute/jobs/execute-property.ts
index 2eff40d12ac..1881d365579 100644
--- a/packages/server/worker/src/lib/execute/jobs/execute-property.ts
+++ b/packages/server/worker/src/lib/execute/jobs/execute-property.ts
@@ -10,7 +10,7 @@ export const executePropertyJob: JobHandler<ExecutePropertyJobData, SynchronousJ
async execute(ctx: JobContext, data: ExecutePropertyJobData): Promise<SynchronousJobResult> {
const timeoutInSeconds = workerSettings.getSettings().TRIGGER_TIMEOUT_SECONDS
- const resolved = await ctx.resolver.resolve({ platformId: data.platformId, publicApiUrl: ctx.publicApiUrl, engineToken: ctx.engineToken, pieces: [data.piece] })
+ const resolved = await ctx.resolver.resolve({ platformId: data.platformId, internalApiUrl: ctx.internalApiUrl, engineToken: ctx.engineToken, pieces: [data.piece] })
if (resolved.kind !== 'ready') {
throw new Error(`Unexpected resolve outcome "${resolved.kind}" for piece-only job`)
}
diff --git a/packages/server/worker/src/lib/execute/jobs/execute-token-refresh.ts b/packages/server/worker/src/lib/execute/jobs/execute-token-refresh.ts
index bd67ef1060d..9e31a4292ba 100644
--- a/packages/server/worker/src/lib/execute/jobs/execute-token-refresh.ts
+++ b/packages/server/worker/src/lib/execute/jobs/execute-token-refresh.ts
@@ -15,7 +15,7 @@ export const executeTokenRefreshJob: JobHandler<ExecuteTokenRefreshJobData, Sync
async execute(ctx: JobContext, data: ExecuteTokenRefreshJobData): Promise<SynchronousJobResult> {
const timeoutInSeconds = workerSettings.getSettings().TRIGGER_TIMEOUT_SECONDS
- const resolved = await ctx.resolver.resolve({ platformId: data.platformId, publicApiUrl: ctx.publicApiUrl, engineToken: ctx.engineToken, pieces: [data.piece] })
+ const resolved = await ctx.resolver.resolve({ platformId: data.platformId, internalApiUrl: ctx.internalApiUrl, engineToken: ctx.engineToken, pieces: [data.piece] })
if (resolved.kind !== 'ready') {
throw new Error(`Unexpected resolve outcome "${resolved.kind}" for piece-only job`)
}
diff --git a/packages/server/worker/src/lib/execute/jobs/execute-trigger-hook.ts b/packages/server/worker/src/lib/execute/jobs/execute-trigger-hook.ts
index 144ab21a9db..449ec457d1a 100644
--- a/packages/server/worker/src/lib/execute/jobs/execute-trigger-hook.ts
+++ b/packages/server/worker/src/lib/execute/jobs/execute-trigger-hook.ts
@@ -11,7 +11,7 @@ export const executeTriggerHookJob: JobHandler<ExecuteTriggerHookJobData, Synchr
async execute(ctx: JobContext, data: ExecuteTriggerHookJobData): Promise<SynchronousJobResult> {
const timeoutInSeconds = workerSettings.getSettings().TRIGGER_HOOKS_TIMEOUT_SECONDS
- const resolved = await ctx.resolver.resolve({ platformId: data.platformId, publicApiUrl: ctx.publicApiUrl, engineToken: ctx.engineToken, flow: { id: data.flowId, versionId: data.flowVersionId, projectId: data.projectId } })
+ const resolved = await ctx.resolver.resolve({ platformId: data.platformId, internalApiUrl: ctx.internalApiUrl, engineToken: ctx.engineToken, flow: { id: data.flowId, versionId: data.flowVersionId, projectId: data.projectId } })
if (resolved.kind === 'flow-not-found') {
ctx.log.info({ flowVersion: { id: data.flowVersionId } }, 'Flow version not found for trigger hook, skipping')
diff --git a/packages/server/worker/src/lib/execute/jobs/execute-validation.ts b/packages/server/worker/src/lib/execute/jobs/execute-validation.ts
index 5a218a44d6c..a61f4ce1a6b 100644
--- a/packages/server/worker/src/lib/execute/jobs/execute-validation.ts
+++ b/packages/server/worker/src/lib/execute/jobs/execute-validation.ts
@@ -9,7 +9,7 @@ export const executeValidationJob: JobHandler<ExecuteValidateAuthJobData, Synchr
async execute(ctx: JobContext, data: ExecuteValidateAuthJobData): Promise<SynchronousJobResult> {
const timeoutInSeconds = workerSettings.getSettings().TRIGGER_TIMEOUT_SECONDS
- const resolved = await ctx.resolver.resolve({ platformId: data.platformId, publicApiUrl: ctx.publicApiUrl, engineToken: ctx.engineToken, pieces: [data.piece] })
+ const resolved = await ctx.resolver.resolve({ platformId: data.platformId, internalApiUrl: ctx.internalApiUrl, engineToken: ctx.engineToken, pieces: [data.piece] })
if (resolved.kind !== 'ready') {
throw new Error(`Unexpected resolve outcome "${resolved.kind}" for piece-only job`)
}
diff --git a/packages/server/worker/src/lib/execute/jobs/execute-webhook.ts b/packages/server/worker/src/lib/execute/jobs/execute-webhook.ts
index cc3b057727b..37eeca55efd 100644
--- a/packages/server/worker/src/lib/execute/jobs/execute-webhook.ts
+++ b/packages/server/worker/src/lib/execute/jobs/execute-webhook.ts
@@ -27,7 +27,7 @@ export const executeWebhookJob: JobHandler<WebhookJobData, FireAndForgetJobResul
const settings = workerSettings.getSettings()
const timeoutInSeconds = settings.TRIGGER_TIMEOUT_SECONDS
- const resolved = await ctx.resolver.resolve({ platformId: data.platformId, publicApiUrl: ctx.publicApiUrl, engineToken: ctx.engineToken, flow: { id: data.flowId, versionId: data.flowVersionIdToRun, projectId: data.projectId } })
+ const resolved = await ctx.resolver.resolve({ platformId: data.platformId, internalApiUrl: ctx.internalApiUrl, engineToken: ctx.engineToken, flow: { id: data.flowId, versionId: data.flowVersionIdToRun, projectId: data.projectId } })
if (resolved.kind === 'flow-not-found') {
ctx.log.info({ flowVersion: { id: data.flowVersionIdToRun } }, 'Flow version not found for webhook, skipping')
diff --git a/packages/server/worker/src/lib/execute/jobs/extract-piece-info.ts b/packages/server/worker/src/lib/execute/jobs/extract-piece-info.ts
index 07b77abfe4a..803dd979cf9 100644
--- a/packages/server/worker/src/lib/execute/jobs/extract-piece-info.ts
+++ b/packages/server/worker/src/lib/execute/jobs/extract-piece-info.ts
@@ -12,7 +12,7 @@ export const extractPieceInfoJob: JobHandler<ExecuteExtractPieceMetadataJobData,
async execute(ctx: JobContext, data: ExecuteExtractPieceMetadataJobData): Promise<SynchronousJobResult> {
const timeoutInSeconds = workerSettings.getSettings().TRIGGER_TIMEOUT_SECONDS
- const resolved = await ctx.resolver.resolve({ platformId: data.platformId, publicApiUrl: ctx.publicApiUrl, engineToken: ctx.engineToken, pieces: [data.piece] })
+ const resolved = await ctx.resolver.resolve({ platformId: data.platformId, internalApiUrl: ctx.internalApiUrl, engineToken: ctx.engineToken, pieces: [data.piece] })
if (resolved.kind !== 'ready') {
throw new Error(`Unexpected resolve outcome "${resolved.kind}" for piece-only job`)
}
diff --git a/packages/server/worker/src/lib/execute/jobs/renew-webhook.ts b/packages/server/worker/src/lib/execute/jobs/renew-webhook.ts
index 354552bd3b0..c4bc1fd4bcb 100644
--- a/packages/server/worker/src/lib/execute/jobs/renew-webhook.ts
+++ b/packages/server/worker/src/lib/execute/jobs/renew-webhook.ts
@@ -9,7 +9,7 @@ export const renewWebhookJob: JobHandler<RenewWebhookJobData, FireAndForgetJobRe
async execute(ctx: JobContext, data: RenewWebhookJobData): Promise<FireAndForgetJobResult> {
const timeoutInSeconds = workerSettings.getSettings().TRIGGER_HOOKS_TIMEOUT_SECONDS
- const resolved = await ctx.resolver.resolve({ platformId: data.platformId, publicApiUrl: ctx.publicApiUrl, engineToken: ctx.engineToken, flow: { id: data.flowId, versionId: data.flowVersionId, projectId: data.projectId } })
+ const resolved = await ctx.resolver.resolve({ platformId: data.platformId, internalApiUrl: ctx.internalApiUrl, engineToken: ctx.engineToken, flow: { id: data.flowId, versionId: data.flowVersionId, projectId: data.projectId } })
if (resolved.kind === 'flow-not-found') {
ctx.log.info({ flowVersion: { id: data.flowVersionId } }, 'Flow version not found for renew webhook, skipping')
diff --git a/packages/server/worker/src/lib/execute/jobs/resolve-connection-identifier.ts b/packages/server/worker/src/lib/execute/jobs/resolve-connection-identifier.ts
index da07ce8edbd..2bb564517d3 100644
--- a/packages/server/worker/src/lib/execute/jobs/resolve-connection-identifier.ts
+++ b/packages/server/worker/src/lib/execute/jobs/resolve-connection-identifier.ts
@@ -8,7 +8,7 @@ export const resolveConnectionIdentifierJob: JobHandler<ExecuteResolveConnection
async execute(ctx: JobContext, data: ExecuteResolveConnectionIdentifierJobData): Promise<SynchronousJobResult> {
const timeoutInSeconds = workerSettings.getSettings().TRIGGER_TIMEOUT_SECONDS
- const resolved = await ctx.resolver.resolve({ platformId: data.platformId, publicApiUrl: ctx.publicApiUrl, engineToken: ctx.engineToken, pieces: [data.piece] })
+ const resolved = await ctx.resolver.resolve({ platformId: data.platformId, internalApiUrl: ctx.internalApiUrl, engineToken: ctx.engineToken, pieces: [data.piece] })
if (resolved.kind !== 'ready') {
throw new Error(`Unexpected resolve outcome "${resolved.kind}" for piece-only job`)
}
diff --git a/packages/server/worker/src/lib/worker.ts b/packages/server/worker/src/lib/worker.ts
index 590cb089528..9e29fc4caf8 100644
--- a/packages/server/worker/src/lib/worker.ts
+++ b/packages/server/worker/src/lib/worker.ts
@@ -123,7 +123,7 @@ export const worker = {
createNotifyServer<ApiToWorkerContract>(socket, createApiToWorkerHandlers({
getRuntime: () => runtime,
apiClient,
- getPublicApiUrl: () => ensurePublicApiUrl(workerSettings.getSettings().PUBLIC_URL),
+ getInternalApiUrl: getApiUrl,
log: logger,
}), logger)
@@ -196,7 +196,7 @@ async function startPollingWorkers(apiClient: WorkerToApiContract): Promise<void
const { error: prewarmError } = await tryCatch(() => createdRuntime.prewarm({
log: logger,
apiClient,
- publicApiUrl: ensurePublicApiUrl(workerSettings.getSettings().PUBLIC_URL),
+ internalApiUrl: getApiUrl(),
}))
if (prewarmError) {
logger.error({ error: prewarmError }, 'Prewarm failed, continuing without a warm cache')
diff --git a/packages/server/worker/test/lib/execute/jobs/execute-flow.test.ts b/packages/server/worker/test/lib/execute/jobs/execute-flow.test.ts
index 8e275308061..2c169e0d7f7 100644
--- a/packages/server/worker/test/lib/execute/jobs/execute-flow.test.ts
+++ b/packages/server/worker/test/lib/execute/jobs/execute-flow.test.ts
@@ -86,7 +86,7 @@ function makeMockContext(opts?: { resolveResult?: unknown, apiOverrides?: Record
resolve: vi.fn().mockResolvedValue(
opts?.resolveResult ?? {
kind: 'ready',
- provision: { platformId: 'plat-1', pieces: [], codes: [], publicApiUrl: 'http://localhost:3000/api/', engineToken: 'test-token' },
+ provision: { platformId: 'plat-1', pieces: [], codes: [], internalApiUrl: 'http://localhost:3000/api/', engineToken: 'test-token' },
flowVersion: makeFlowVersion(),
},
),
@@ -109,7 +109,7 @@ function makeMockContext(opts?: { resolveResult?: unknown, apiOverrides?: Record
runtime,
workerIndex: 0,
engineToken: 'test-token',
- internalApiUrl: 'http://localhost:3000',
+ internalApiUrl: 'http://localhost:3000/api/',
publicApiUrl: 'http://localhost:4200',
} as any
}
@@ -152,6 +152,18 @@ describe('executeFlowJob', () => {
})
})
+ describe('piece bundle source', () => {
+ it('resolves pieces against the worker internal API URL, not the public one', async () => {
+ const ctx = makeMockContext()
+
+ await executeFlowJob.execute(ctx, makeResumeJobData({ executionType: ExecutionType.BEGIN }))
+
+ const resolveInput = ctx.resolver.resolve.mock.calls[0][0]
+ expect(resolveInput.internalApiUrl).toBe('http://localhost:3000/api/')
+ expect(resolveInput).not.toHaveProperty('publicApiUrl')
+ })
+ })
+
describe('RESUME validation', () => {
it('still throws when logsFileId is missing for RESUME', async () => {
const ctx = makeMockContext()