AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents
TypeScript ai-agentai-agent-toolsai-agentsai-agents-frameworkmcpmcp-servermcp-toolsmcpsn8n-alternativeno-code-automationworkflowworkflow-automationworkflows
feat(event-streaming): gate delivery on the plan, add an enabled flag and OTLP log formats (#15872)
Othman Abu Ajamiehf27cf8fcd902
diff --git a/brain/knowledge/eventing-webhooks/event-destinations.md b/brain/knowledge/eventing-webhooks/event-destinations.mdindex 52dd73d9341..215e9b7ace5 100644--- a/brain/knowledge/eventing-webhooks/event-destinations.md+++ b/brain/knowledge/eventing-webhooks/event-destinations.md@@ -4,20 +4,25 @@ icon: 📡# Event Destinations-Streams platform/project activity events to webhook URLs in real time. Internal Activepieces flow webhooks are valid targets, so operators can route events into a flow and fan out to Slack/Gmail/Teams/HTTP without leaving the platform. EE/Cloud only, gated by `auditLogEnabled` (shares audit-log gating). Lives under the **Observability** sidebar group.+Streams platform/project activity events to webhook URLs in real time. Internal Activepieces flow webhooks are valid targets, so operators can route events into a flow and fan out to Slack/Gmail/Teams/HTTP without leaving the platform. EE/Cloud only, gated by `eventStreamingEnabled` on both the API (402 preHandler) and delivery (`eventDestinationHooks`). Lives on the **Security** admin page as the `events` tab (`/platform/security?tab=events`).### Entities & services- **EventDestination**: one URL receiving a chosen subset of the 27 `ApplicationEventName` events, at PLATFORM or PROJECT scope.- **Scope**: PLATFORM (all platform events) or PROJECT (currently only `FLOW_RUN_FINISHED`).+- **Format**: how a destination encodes each event on the wire — `RAW` (the event JSON, the default and what every pre-format destination sends), `OTLP_JSON`, or `OTLP_PROTOBUF` (an OTLP/HTTP logs `ExportLogsServiceRequest`, built by `otlpLogs` in `@activepieces/server-utils` on top of evlog's `toOTLPLogRecord`).- Delivery via BullMQ queue (`WorkerJobType.EVENT_DESTINATION`).### How it works- `eventDestinationService.trigger()` finds matching destinations and classifies each URL as internal vs external.-- **Internal** (URL origin == instance's public API origin, path under `/v1/webhooks/`, suffix `''` or `/sync`): dispatched directly via `webhookService.handleWebhook` — no outbound HTTP, so the SSRF filter never sees a self-referential private-IP call (GIT-1539). `/draft` and `/test` URLs stay on the outbound path.+- **Internal** (URL origin == instance's public API origin, path under `/v1/webhooks/<flowId>`, any route suffix): dispatched directly via `webhookService.handleWebhook` — no outbound HTTP, so the SSRF filter never sees a self-referential private-IP call (GIT-1539). A `/draft` URL runs the latest version; a `/test` URL only saves sample data and does not run the flow.- **External**: queued as ONE_TIME jobs, POSTed via `safeHttp` (SSRF-protected); delivery failures logged at error level.-- Endpoints: `POST/GET/DELETE /v1/event-destinations` (+ `POST /:id`), and `POST /test` sends a mock event (defaults to `FLOW_CREATED`) built by `buildMockEvent()`.+- Endpoints: `POST/GET/DELETE /v1/event-destinations` (+ `POST /:id` and `PATCH /:id`), and `POST /test` sends a mock event (defaults to `FLOW_CREATED`, format `RAW`) built by `buildMockEvent()`.### Gotchas+- **An OTLP destination URL is used exactly as pasted.** The OTLP spec says a signal-specific endpoint "MUST be used as-is", and vendor paths differ (`/v1/logs`, Loki `/otlp/v1/logs`, PostHog `/i/v1/logs`), so nothing appends `/v1/logs`.+- **The worker encodes protobuf, not the API**, because BullMQ stores job data as JSON and raw bytes do not survive the queue. The queued payload is always the OTLP/JSON object; `contentType: 'application/x-protobuf'` on the job tells the worker to encode it. A job without `contentType` posts JSON, which is exactly the pre-format job shape.+- **The API refuses `OTLP_PROTOBUF` for any webhook URL** (400, `EVENT_DESTINATION_FORMAT_NOT_SUPPORTED_BY_WEBHOOK`), on create and on update. The webhook routes have no protobuf parser, so an HTTP POST of protobuf gets a 415, and the internal dispatch would hand the flow JSON anyway. The check uses `extractWebhookFlowIdCandidate`, host-agnostic like the cycle guard, so an embed-host URL is refused too. `OTLP_JSON` stays allowed: both paths deliver it as JSON.+- **The OTLP attribute keys are a customer contract.** Envelope fields keep their RAW names, and `data` is flattened to dotted keys (`data.flowRun.status`); the body is the raw event JSON, not evlog's wide event. `packages/server/utils/test/otlp-logs.test.ts` pins the whole request, so an evlog upgrade that changes `toOTLPLogRecord` fails a test instead of silently changing every customer's dashboards.- The old `assertUrlIsExternal` check was **removed** to support internal handler flows; recursion is instead prevented by a server-side **cycle guard** keyed on the target flow of any webhook URL, regardless of route suffix (so a self-targeting `/draft` destination is still dropped on its own run events).- **The cycle guard is host-agnostic on purpose — do not "tighten" it back to an origin check.** It matches any destination whose path contains `/v1/webhooks/<flowId>`, percent-decoding the id segment. It used to require `destination.origin === getPublicApiUrl().origin`, which is `AP_FRONTEND_URL`: platforms with an embed subdomain get `WEBHOOK_URL_PREFIX` rewritten to `https://<embed-hostname>/api/v1/webhooks`, so the origin never matched, the guard failed open, and a handler flow re-triggered on its own runs — one webhook POST produced 838 runs in ~3 min (GIT-1641, Pylon 5378). `domainHelper.getPublicApiUrl()` is called with no `platformId` here, so it can never resolve to the embed host. The path is the stable part (`/v1/webhooks` is fixed by the route registration and every URL builder); the host is not.- **When the guard fires it drops *every* webhook-shaped destination for that event, not just the self-target.** Exact-match filtering would let two flows wired to each other ping-pong forever — destinations are platform/project-scoped, so the mutually-wired pair both sit in the same candidate list. The cost is that genuine cross-instance chaining (a destination pointing at *another* instance's `/v1/webhooks/...`) is also dropped for the looping flow's events; the `droppedDestinations` field on the cycle-break `warn` is how you see that happened.@@ -31,10 +36,11 @@ Entry point: `eventDestinationService`, a log-scoped factory wired up in `platfo- `packages/server/api/src/app/event-destinations/` — the service (trigger, internal vs external routing, cycle guard) and the typeorm entity- `packages/server/api/src/app/ee/platform-webhooks/` — the HTTP controller and module that expose the endpoints-- `packages/server/worker/src/lib/execute/jobs/event-destination.ts` — worker side of the BullMQ job, POSTs external destinations via safeHttp+- `packages/server/worker/src/lib/execute/jobs/event-destination.ts` — worker side of the BullMQ job, POSTs external destinations via safeHttp and encodes protobuf+- `packages/server/utils/src/` — `otlpLogs`, the OTLP/JSON builder and the inline protobuf descriptor (field numbers from `opentelemetry-proto` v1.11.0)- `packages/core/shared/src/lib/ee/event-destinations/` — request/response zod schemas and barrel export- `packages/core/shared/src/lib/ee/audit-events/` — the `ApplicationEventName` enum plus `buildMockEvent()` for test deliveries- `packages/web/src/app/routes/platform/infra/event-destinations/` — the page, its TanStack DB collection, handler-flow builder, and dialog/row/action components- `packages/server/api/test/integration/cloud/event-destinations/` — integration tests covering CRUD and trigger dispatch-Paths verified 2026-07-17.+Paths verified 2026-09-23.diff --git a/brain/knowledge/eventing-webhooks/index.md b/brain/knowledge/eventing-webhooks/index.mdindex b3af7d627b2..b1c554e5ae7 100644--- a/brain/knowledge/eventing-webhooks/index.md+++ b/brain/knowledge/eventing-webhooks/index.md@@ -17,8 +17,16 @@ Ownership verification a provider requires before it will send events — a chalAn internal-bus domain event (27 names). The one vocabulary shared by audit logs, event destinations, and alerts — not an HTTP concept.### 📡 Event Destination-An outbound webhook subscribing to a chosen subset of Application Events at PLATFORM or PROJECT scope, delivered via BullMQ. EE/Cloud only, gated by `auditLogEnabled`.-- *Avoid:* "event stream" — the delivery is per-destination fan-out, not a stream anyone subscribes to.+An outbound endpoint subscribing to a chosen subset of Application Events at PLATFORM or PROJECT scope, delivered via BullMQ in its Format. EE/Cloud only, gated by `eventStreamingEnabled`. "Event Streaming" is only the name of the feature and its admin page.+- *Avoid:* "event stream", "audit stream" — the delivery is per-destination fan-out, not a stream anyone subscribes to.++### 🧾 Format+How an Event Destination encodes each Application Event on the wire: `RAW` (the event JSON), `OTLP_JSON`, or `OTLP_PROTOBUF` (an OpenTelemetry logs export request).+- *Avoid:* "mapper", "payload template", "preset", "destination type" — all retired; the encoding is part of the Format, not a separate setting.++### 🧭 Handler Flow+A flow on the same instance whose webhook URL is an Event Destination URL. It gets events in-process when the URL is on the instance's public origin, and over HTTP otherwise (an embed-host URL). A generated one routes on the `RAW` Format; the API refuses `OTLP_PROTOBUF` for any webhook URL.+- *Avoid:* "internal flow".## Pagesdiff --git a/bun.lock b/bun.lockindex 1c37066b2a9..5d687e483ef 100644--- a/bun.lock+++ b/bun.lock@@ -120,7 +120,7 @@},"packages/core/execution": {"name": "@activepieces/core-execution",- "version": "0.24.0",+ "version": "0.25.0","dependencies": {"@activepieces/core-piece-types": "workspace:*","@activepieces/core-utils": "workspace:*",@@ -165,7 +165,7 @@},"packages/core/shared": {"name": "@activepieces/shared",- "version": "0.192.0",+ "version": "0.193.0","dependencies": {"@activepieces/core-execution": "workspace:*","@activepieces/core-formula": "workspace:*",@@ -188,7 +188,7 @@},"packages/core/utils": {"name": "@activepieces/core-utils",- "version": "0.9.0",+ "version": "0.10.0","dependencies": {"deepmerge-ts": "8.0.2","ipaddr.js": "2.3.0",@@ -11008,6 +11008,7 @@"dayjs": "1.11.9","evlog": "2.22.4","fastify": "5.12.5",+ "protobufjs": "7.6.6","request-filtering-agent": "3.2.1","systeminformation": "5.31.7","tslib": "2.6.2",@@ -13858,9 +13859,9 @@"@protobufjs/codegen": ["@protobufjs/[email protected]", "", {}, "sha512-zgXFLzW3Ap33e6d0Wlj4MGIm6Ce8O89n/apUaGNB/jx+hw+ruWEp7EwGUshdLKVRCxZW12fp9r40E1mQrf/34g=="],- "@protobufjs/eventemitter": ["@protobufjs/[email protected]", "", {}, "sha512-j9ednRT81vYJ9OfVuXG6ERSTdEL1xVsNgqpkxMsbIabzSo3goCjDIveeGv5d03om39ML71RdmrGNjG5SReBP/Q=="],+ "@protobufjs/eventemitter": ["@protobufjs/[email protected]", "", {}, "sha512-vW1GmwMZNnL+gMRaovlh9yZX74kc+TTU3FObkkurpMaRtBfLP3ldjS9KQWlwZgraRE0+dheEEoAxdzcJQ8eXZg=="],- "@protobufjs/fetch": ["@protobufjs/[email protected]", "", { "dependencies": { "@protobufjs/aspromise": "^1.1.1", "@protobufjs/inquire": "^1.1.0" } }, "sha512-lljVXpqXebpsijW71PZaCYeIcE5on1w5DlQy5WH6GLbFryLUrBD4932W/E2BSpfRJWseIL4v/KPgBFxDOIdKpQ=="],+ "@protobufjs/fetch": ["@protobufjs/[email protected]", "", { "dependencies": { "@protobufjs/aspromise": "^1.1.1" } }, "sha512-GpptLrs57adMSuHi3VNj0mAF8dwh36LMaYF6XyJ6JMWlVsc+t42tm1HSEDmOs3A8fC9yyeisgLhsTVQokOZ0zw=="],"@protobufjs/float": ["@protobufjs/[email protected]", "", {}, "sha512-Ddb+kVXlXst9d+R9PfTIxh1EdNkgoRe5tOX6t01f1lYWOvJnSPDBlG241QLzcyPdoNTsblLUdujGSE4RzrTZGQ=="],@@ -16540,7 +16541,7 @@"loglevel": ["[email protected]", "", {}, "sha512-HgMmCqIJSAKqo68l0rS2AanEWfkxaZ5wNiEFb5ggm08lDs9Xl2KxBlX3PTcaD2chBM1gXAYf491/M2Rv8Jwayg=="],- "long": ["[email protected]", "", {}, "sha512-XsP+KhQif4bjX1kbuSiySJFNAehNxgLb6hPRGJ9QsUr8ajHkuXGdrHmFUTUUXhDwVX2R5bY4JNZEwbUiMhV+MA=="],+ "long": ["[email protected]", "", {}, "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA=="],"longest-streak": ["[email protected]", "", {}, "sha512-vM6rUVCVUJJt33bnmHiZEvr7wPT78ztX7rojL+LW51bHtLh6HTjx84LA5W4+oa6aKEJA7jJu5LR6vQRBpA5DVg=="],@@ -17196,7 +17197,7 @@"proto-list": ["[email protected]", "", {}, "sha512-vtK/94akxsTMhe0/cbfpR+syPuszcuwhqVjJq26CuNDgFGj682oRBXOP5MJpv2r7JtE8MsiepGIqvvOTBwn2vA=="],- "protobufjs": ["[email protected]", "", { "dependencies": { "@protobufjs/aspromise": "^1.1.2", "@protobufjs/base64": "^1.1.2", "@protobufjs/codegen": "^2.0.4", "@protobufjs/eventemitter": "^1.1.0", "@protobufjs/fetch": "^1.1.0", "@protobufjs/float": "^1.0.2", "@protobufjs/inquire": "^1.1.0", "@protobufjs/path": "^1.1.2", "@protobufjs/pool": "^1.1.0", "@protobufjs/utf8": "^1.1.0", "@types/long": "^4.0.1", "@types/node": ">=13.7.0", "long": "^4.0.0" }, "bin": { "pbjs": "bin/pbjs", "pbts": "bin/pbts" } }, "sha512-OKjVH3hDoXdIZ/s5MLv8O2X0s+wOxGfV7ar6WFSKGaSAxi/6gYn3px5POS4vi+mc/0zCOdL7Jkwrj0oT1Yst2A=="],+ "protobufjs": ["[email protected]", "", { "dependencies": { "@protobufjs/aspromise": "^1.1.2", "@protobufjs/base64": "^1.1.2", "@protobufjs/codegen": "^2.0.5", "@protobufjs/eventemitter": "^1.1.1", "@protobufjs/fetch": "^1.1.1", "@protobufjs/float": "^1.0.2", "@protobufjs/path": "^1.1.2", "@protobufjs/pool": "^1.1.0", "@protobufjs/utf8": "^1.1.1", "@types/node": ">=13.7.0", "long": "^5.3.2" } }, "sha512-dYDWdjSl5RNb7SgPxGQcRU+GtvP7s2fpkrY0r432PcOIaZ0/rBcxEZnQN67iJhFuQiVw754JDoPruPCNdGsbjg=="],"proxy-addr": ["[email protected]", "", { "dependencies": { "forwarded": "0.2.0", "ipaddr.js": "1.9.1" } }, "sha512-llQsMLSUDUPT44jdrU/O37qlnifitDP+ZwrmmZcoSKyLKvtZxpyV0n2/bD/N4tBAAZ/gJEdZU7KMraoK1+XYAg=="],@@ -19440,6 +19441,8 @@"@grpc/grpc-js/@grpc/proto-loader": ["@grpc/[email protected]", "", { "dependencies": { "lodash.camelcase": "^4.3.0", "long": "^5.0.0", "protobufjs": "^7.5.3", "yargs": "^17.7.2" }, "bin": { "proto-loader-gen-types": "build/bin/proto-loader-gen-types.js" } }, "sha512-rc1hOQtjIWGxcxpb9aHAfLpIctjEnsDehj0DAiVfBlmT84uvR0uUtN2hEi/ecvWVjXUGf5qPF4qEgiLOx1YIMQ=="],+ "@grpc/proto-loader/protobufjs": ["[email protected]", "", { "dependencies": { "@protobufjs/aspromise": "^1.1.2", "@protobufjs/base64": "^1.1.2", "@protobufjs/codegen": "^2.0.4", "@protobufjs/eventemitter": "^1.1.0", "@protobufjs/fetch": "^1.1.0", "@protobufjs/float": "^1.0.2", "@protobufjs/inquire": "^1.1.0", "@protobufjs/path": "^1.1.2", "@protobufjs/pool": "^1.1.0", "@protobufjs/utf8": "^1.1.0", "@types/long": "^4.0.1", "@types/node": ">=13.7.0", "long": "^4.0.0" }, "bin": { "pbjs": "bin/pbjs", "pbts": "bin/pbts" } }, "sha512-OKjVH3hDoXdIZ/s5MLv8O2X0s+wOxGfV7ar6WFSKGaSAxi/6gYn3px5POS4vi+mc/0zCOdL7Jkwrj0oT1Yst2A=="],+"@hubspot/api-client/form-data": ["[email protected]", "", { "dependencies": { "asynckit": "^0.4.0", "combined-stream": "^1.0.8", "es-set-tostringtag": "^2.1.0", "hasown": "^2.0.2", "mime-types": "^2.1.35", "safe-buffer": "^5.2.1" } }, "sha512-jqdObeR2rxZZbPSGL+3VckHMYtu+f9//KXBsVny6JSX/pa38Fy+bGjuG8eW/H6USNQWhLi8Num++cU2yOCNz4A=="],"@img/sharp-wasm32/@emnapi/runtime": ["@emnapi/[email protected]", "", { "dependencies": { "tslib": "^2.4.0" } }, "sha512-vgj7R3y3Wgx24IQaGPA/R6YFXLHVMOZ0uVEyIQPaWs+rd1AzfEMXlAC22FYwO1XkKR6NPsq7mUandH8oIRdZFw=="],@@ -19468,8 +19471,6 @@"@mailchain/crypto/@noble/curves": ["@noble/[email protected]", "", { "dependencies": { "@noble/hashes": "1.3.0" } }, "sha512-OqaOf4RWDaCRuBKJLDURrgVxjLmneGsiCXGuzYB5y95YithZMA6w4uk34DHSm0rKMrrYiaeZj48/81EvaAScLQ=="],- "@mailchain/internal/long": ["[email protected]", "", {}, "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA=="],-"@mailchain/internal/protobufjs": ["[email protected]", "", { "dependencies": { "@protobufjs/aspromise": "^1.1.2", "@protobufjs/base64": "^1.1.2", "@protobufjs/codegen": "^2.0.5", "@protobufjs/eventemitter": "^1.1.0", "@protobufjs/fetch": "^1.1.0", "@protobufjs/float": "^1.0.2", "@protobufjs/inquire": "^1.1.1", "@protobufjs/path": "^1.1.2", "@protobufjs/pool": "^1.1.0", "@protobufjs/utf8": "^1.1.1", "@types/node": ">=13.7.0", "long": "^5.0.0" } }, "sha512-M71sTMB146U3u0di3yup8iM+zv8yPRNQVr1KK4tyBitl3qFvEGucq/rGDRShD2rsJhtN02RJaJ7j5X5hmy8SJg=="],"@mailchain/message-composer/date-fns": ["[email protected]", "", { "dependencies": { "@babel/runtime": "^7.21.0" } }, "sha512-fnULvOpxnC5/Vg3NCiWelDsLiUc9bRwAPs/+LfTLNvetFCtCTN+yQz15C/fs4AwX1R9K5GLtLfn8QW+dWisaAw=="],@@ -20018,6 +20019,8 @@"chromium-bidi/zod": ["[email protected]", "", {}, "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ=="],+ "clarifai-nodejs-grpc/protobufjs": ["[email protected]", "", { "dependencies": { "@protobufjs/aspromise": "^1.1.2", "@protobufjs/base64": "^1.1.2", "@protobufjs/codegen": "^2.0.4", "@protobufjs/eventemitter": "^1.1.0", "@protobufjs/fetch": "^1.1.0", "@protobufjs/float": "^1.0.2", "@protobufjs/inquire": "^1.1.0", "@protobufjs/path": "^1.1.2", "@protobufjs/pool": "^1.1.0", "@protobufjs/utf8": "^1.1.0", "@types/long": "^4.0.1", "@types/node": ">=13.7.0", "long": "^4.0.0" }, "bin": { "pbjs": "bin/pbjs", "pbts": "bin/pbts" } }, "sha512-OKjVH3hDoXdIZ/s5MLv8O2X0s+wOxGfV7ar6WFSKGaSAxi/6gYn3px5POS4vi+mc/0zCOdL7Jkwrj0oT1Yst2A=="],+"clean-css/source-map": ["[email protected]", "", {}, "sha512-UjgapumWlbMhkBgzT7Ykc5YXUT46F0iKu8SGXq0bcwP5dz/h0Plj6enJqjz1Zbq2l5WaqYnrVbwWOWMyF3F47g=="],"cli-truncate/string-width": ["[email protected]", "", { "dependencies": { "emoji-regex": "^10.3.0", "get-east-asian-width": "^1.0.0", "strip-ansi": "^7.1.0" } }, "sha512-tsaTIkKW9b4N+AEj+SVA+WhJzV7/zMhcSu78mLKWSk7cXMOSHsBKFWUs0fWwq8QyK3MgJBQRX6Gbi4kYbdvGkQ=="],@@ -21286,10 +21289,14 @@"@google/genai/google-auth-library/jws": ["[email protected]", "", { "dependencies": { "jwa": "^2.0.1", "safe-buffer": "^5.0.1" } }, "sha512-EKI/M/yqPncGUUh44xz0PxSidXFr/+r0pA70+gIYhjv+et7yxM+s29Y+VGDkovRofQem0fs7Uvf4+YmAdyRduA=="],- "@grpc/grpc-js/@grpc/proto-loader/long": ["[email protected]", "", {}, "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA=="],-"@grpc/grpc-js/@grpc/proto-loader/protobufjs": ["[email protected]", "", { "dependencies": { "@protobufjs/aspromise": "^1.1.2", "@protobufjs/base64": "^1.1.2", "@protobufjs/codegen": "^2.0.5", "@protobufjs/eventemitter": "^1.1.0", "@protobufjs/fetch": "^1.1.0", "@protobufjs/float": "^1.0.2", "@protobufjs/inquire": "^1.1.1", "@protobufjs/path": "^1.1.2", "@protobufjs/pool": "^1.1.0", "@protobufjs/utf8": "^1.1.1", "@types/node": ">=13.7.0", "long": "^5.0.0" } }, "sha512-M71sTMB146U3u0di3yup8iM+zv8yPRNQVr1KK4tyBitl3qFvEGucq/rGDRShD2rsJhtN02RJaJ7j5X5hmy8SJg=="],+ "@grpc/proto-loader/protobufjs/@protobufjs/eventemitter": ["@protobufjs/[email protected]", "", {}, "sha512-j9ednRT81vYJ9OfVuXG6ERSTdEL1xVsNgqpkxMsbIabzSo3goCjDIveeGv5d03om39ML71RdmrGNjG5SReBP/Q=="],++ "@grpc/proto-loader/protobufjs/@protobufjs/fetch": ["@protobufjs/[email protected]", "", { "dependencies": { "@protobufjs/aspromise": "^1.1.1", "@protobufjs/inquire": "^1.1.0" } }, "sha512-lljVXpqXebpsijW71PZaCYeIcE5on1w5DlQy5WH6GLbFryLUrBD4932W/E2BSpfRJWseIL4v/KPgBFxDOIdKpQ=="],++ "@grpc/proto-loader/protobufjs/long": ["[email protected]", "", {}, "sha512-XsP+KhQif4bjX1kbuSiySJFNAehNxgLb6hPRGJ9QsUr8ajHkuXGdrHmFUTUUXhDwVX2R5bY4JNZEwbUiMhV+MA=="],+"@img/sharp-wasm32/@emnapi/runtime/tslib": ["[email protected]", "", {}, "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w=="],"@isaacs/cliui/strip-ansi/ansi-regex": ["[email protected]", "", {}, "sha512-Bq3SmSpyFHaWjPk8If9yc6svM8c56dB5BAtW4Qbw5jHTwwXXcTLoRMkpDJp6VL0XzlWaCHTXrkFURMYmD0sLqg=="],@@ -21298,6 +21305,10 @@"@mailchain/crypto/@noble/curves/@noble/hashes": ["@noble/[email protected]", "", {}, "sha512-ilHEACi9DwqJB0pw7kv+Apvh50jiiSyR/cQ3y4W7lOR5mhvn/50FLUfsnfJz0BDZtl/RR16kXvptiv6q1msYZg=="],+ "@mailchain/internal/protobufjs/@protobufjs/eventemitter": ["@protobufjs/[email protected]", "", {}, "sha512-j9ednRT81vYJ9OfVuXG6ERSTdEL1xVsNgqpkxMsbIabzSo3goCjDIveeGv5d03om39ML71RdmrGNjG5SReBP/Q=="],++ "@mailchain/internal/protobufjs/@protobufjs/fetch": ["@protobufjs/[email protected]", "", { "dependencies": { "@protobufjs/aspromise": "^1.1.1", "@protobufjs/inquire": "^1.1.0" } }, "sha512-lljVXpqXebpsijW71PZaCYeIcE5on1w5DlQy5WH6GLbFryLUrBD4932W/E2BSpfRJWseIL4v/KPgBFxDOIdKpQ=="],+"@npmcli/move-file/rimraf/glob": ["[email protected]", "", { "dependencies": { "fs.realpath": "^1.0.0", "inflight": "^1.0.4", "inherits": "2", "minimatch": "^3.1.1", "once": "^1.3.0", "path-is-absolute": "^1.0.0" } }, "sha512-nFR0zLpU2YCaRxwoCJvL6UvCH2JFyFVIvwTLsIf21AuHlMskA1hhTdk+LlYJtOlYt9v6dvszD2BGRqBL+iQK9Q=="],"@oclif/core/minimatch/brace-expansion": ["[email protected]", "", { "dependencies": { "balanced-match": "^4.0.2" } }, "sha512-VZznLgtwhn+Mact9tfiwx64fA9erHH/MCXEUfB/0bX/6Fz6ny5EGTXYltMocqg4xFAQZtnO3DHWWXi8RiuN7cQ=="],@@ -21408,6 +21419,12 @@"checkly/open/define-lazy-prop": ["[email protected]", "", {}, "sha512-Ds09qNh8yw3khSjiJjiUInaGX9xlqZDY7JVryGxdxV7NPeuqQfplOpQ66yJFZut3jLa5zOwkXw1g9EI2uKh4Og=="],+ "clarifai-nodejs-grpc/protobufjs/@protobufjs/eventemitter": ["@protobufjs/[email protected]", "", {}, "sha512-j9ednRT81vYJ9OfVuXG6ERSTdEL1xVsNgqpkxMsbIabzSo3goCjDIveeGv5d03om39ML71RdmrGNjG5SReBP/Q=="],++ "clarifai-nodejs-grpc/protobufjs/@protobufjs/fetch": ["@protobufjs/[email protected]", "", { "dependencies": { "@protobufjs/aspromise": "^1.1.1", "@protobufjs/inquire": "^1.1.0" } }, "sha512-lljVXpqXebpsijW71PZaCYeIcE5on1w5DlQy5WH6GLbFryLUrBD4932W/E2BSpfRJWseIL4v/KPgBFxDOIdKpQ=="],++ "clarifai-nodejs-grpc/protobufjs/long": ["[email protected]", "", {}, "sha512-XsP+KhQif4bjX1kbuSiySJFNAehNxgLb6hPRGJ9QsUr8ajHkuXGdrHmFUTUUXhDwVX2R5bY4JNZEwbUiMhV+MA=="],+"cli-truncate/string-width/emoji-regex": ["[email protected]", "", {}, "sha512-toUI84YS5YmxW219erniWD0CIVOo46xGKColeNQRgOzDorgBi1v4D71/OFzgD9GO2UGKIv1C3Sp8DAn0+j5w7A=="],"cli-truncate/string-width/strip-ansi": ["[email protected]", "", { "dependencies": { "ansi-regex": "^6.2.2" } }, "sha512-yDPMNjp4WyfYBkHnjIRLfca1i6KMyGCtsVgoKe/z1+6vukgaENdgGBZt+ZmKPc4gavvEZ5OgHfHdrazhgNyG7w=="],@@ -22162,7 +22179,9 @@"@activepieces/piece-google-slides/google-auth-library/jws/jwa": ["[email protected]", "", { "dependencies": { "buffer-equal-constant-time": "^1.0.1", "ecdsa-sig-formatter": "1.0.11", "safe-buffer": "^5.0.1" } }, "sha512-hRF04fqJIP8Abbkq5NKGN0Bbr3JxlQ+qhZufXVr0DvujKy93ZCbXZMHDL4EOtodSbCWxOqR8MS1tXA5hwqCXDg=="],- "@activepieces/piece-google-vertexai/@google/genai/protobufjs/long": ["[email protected]", "", {}, "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA=="],+ "@activepieces/piece-google-vertexai/@google/genai/protobufjs/@protobufjs/eventemitter": ["@protobufjs/[email protected]", "", {}, "sha512-j9ednRT81vYJ9OfVuXG6ERSTdEL1xVsNgqpkxMsbIabzSo3goCjDIveeGv5d03om39ML71RdmrGNjG5SReBP/Q=="],++ "@activepieces/piece-google-vertexai/@google/genai/protobufjs/@protobufjs/fetch": ["@protobufjs/[email protected]", "", { "dependencies": { "@protobufjs/aspromise": "^1.1.1", "@protobufjs/inquire": "^1.1.0" } }, "sha512-lljVXpqXebpsijW71PZaCYeIcE5on1w5DlQy5WH6GLbFryLUrBD4932W/E2BSpfRJWseIL4v/KPgBFxDOIdKpQ=="],"@activepieces/piece-google-vertexai/google-auth-library/gaxios/node-fetch": ["[email protected]", "", { "dependencies": { "data-uri-to-buffer": "^4.0.0", "fetch-blob": "^3.1.4", "formdata-polyfill": "^4.0.10" } }, "sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA=="],@@ -22316,6 +22335,10 @@"@google/genai/google-auth-library/jws/jwa": ["[email protected]", "", { "dependencies": { "buffer-equal-constant-time": "^1.0.1", "ecdsa-sig-formatter": "1.0.11", "safe-buffer": "^5.0.1" } }, "sha512-hRF04fqJIP8Abbkq5NKGN0Bbr3JxlQ+qhZufXVr0DvujKy93ZCbXZMHDL4EOtodSbCWxOqR8MS1tXA5hwqCXDg=="],+ "@grpc/grpc-js/@grpc/proto-loader/protobufjs/@protobufjs/eventemitter": ["@protobufjs/[email protected]", "", {}, "sha512-j9ednRT81vYJ9OfVuXG6ERSTdEL1xVsNgqpkxMsbIabzSo3goCjDIveeGv5d03om39ML71RdmrGNjG5SReBP/Q=="],++ "@grpc/grpc-js/@grpc/proto-loader/protobufjs/@protobufjs/fetch": ["@protobufjs/[email protected]", "", { "dependencies": { "@protobufjs/aspromise": "^1.1.1", "@protobufjs/inquire": "^1.1.0" } }, "sha512-lljVXpqXebpsijW71PZaCYeIcE5on1w5DlQy5WH6GLbFryLUrBD4932W/E2BSpfRJWseIL4v/KPgBFxDOIdKpQ=="],+"@oclif/core/minimatch/brace-expansion/balanced-match": ["[email protected]", "", {}, "sha512-BLrgEcRTwX2o6gGxGOCNyMvGSp35YofuYzw9h1IMTRmKqttAZZVU67bdb9Pr2vUHA8+j3i2tJfjO6C6+4myGTA=="],"@rolldown/binding-wasm32-wasi/@napi-rs/wasm-runtime/@emnapi/core/@emnapi/wasi-threads": ["@emnapi/[email protected]", "", { "dependencies": { "tslib": "^2.4.0" } }, "sha512-uTII7OYF+/Mes/MrcIOYp5yOtSMLBWSIoLPpcgwipoiKbli6k322tcoFsxoIIxPDqW01SQGAgko4EzZi2BNv2w=="],diff --git a/packages/core/execution/package.json b/packages/core/execution/package.jsonindex b46da93b4f7..d14093bb61d 100644--- a/packages/core/execution/package.json+++ b/packages/core/execution/package.json@@ -1,6 +1,6 @@{"name": "@activepieces/core-execution",- "version": "0.24.0",+ "version": "0.25.0","type": "commonjs","main": "./dist/src/index.js","scripts": {diff --git a/packages/core/execution/src/lib/workers/job-data.ts b/packages/core/execution/src/lib/workers/job-data.tsindex e77fba85e11..05ef831f1ed 100644--- a/packages/core/execution/src/lib/workers/job-data.ts+++ b/packages/core/execution/src/lib/workers/job-data.ts@@ -365,6 +365,7 @@ export const EventDestinationJobData = z.object({webhookId: z.string(),webhookUrl: z.string(),payload: z.unknown(), // EE audit-event schema stays in @activepieces/shared; engine does not need it+ contentType: z.enum(['application/json', 'application/x-protobuf']).optional(),jobType: z.literal(WorkerJobType.EVENT_DESTINATION),})diff --git a/packages/core/shared/package.json b/packages/core/shared/package.jsonindex b9cadc19473..173477c5ab2 100644--- a/packages/core/shared/package.json+++ b/packages/core/shared/package.json@@ -1,6 +1,6 @@{"name": "@activepieces/shared",- "version": "0.192.0",+ "version": "0.193.0","type": "commonjs","sideEffects": false,"main": "./dist/src/index.js",diff --git a/packages/core/shared/src/lib/ee/audit-events/mock-event-builder.ts b/packages/core/shared/src/lib/ee/audit-events/mock-event-builder.tsindex faeb0df47e5..6152707c391 100644--- a/packages/core/shared/src/lib/ee/audit-events/mock-event-builder.ts+++ b/packages/core/shared/src/lib/ee/audit-events/mock-event-builder.ts@@ -30,16 +30,27 @@ import {export const buildMockEvent = ({ event, platformId, projectId }: BuildMockEventParams): ApplicationEvent => {const isoNow = new Date().toISOString()- const baseEnvelope = {+ const project = { displayName: 'Dream Department' }+ const user = {+ id: apId(),+ email: '[email protected]',+ firstName: 'Sample',+ lastName: 'User',+ }+ const workerEnvelope = {id: apId(),created: isoNow,updated: isoNow,- ip: '127.0.0.1',platformId,projectId,- userId: apId(),}- const project = { displayName: 'Dream Department' }+ const baseEnvelope = {+ ...workerEnvelope,+ ip: '127.0.0.1',+ userId: user.id,+ userEmail: user.email,+ projectDisplayName: project.displayName,+ }const flow = { id: apId(), externalId: apId(), created: isoNow, updated: isoNow }const flowVersion = {id: apId(),@@ -48,12 +59,6 @@ export const buildMockEvent = ({ event, platformId, projectId }: BuildMockEventPcreated: isoNow,updated: isoNow,}- const user = {- id: apId(),- email: '[email protected]',- firstName: 'Sample',- lastName: 'User',- }switch (event) {case ApplicationEventName.FLOW_RUN_STARTED:@@ -61,7 +66,7 @@ export const buildMockEvent = ({ event, platformId, projectId }: BuildMockEventPcase ApplicationEventName.FLOW_RUN_RESUMED:case ApplicationEventName.FLOW_RUN_RETRIED: {const mock: FlowRunEvent = {- ...baseEnvelope,+ ...workerEnvelope,action: event,data: {flowRun: {diff --git a/packages/core/shared/src/lib/ee/event-destinations/dto.ts b/packages/core/shared/src/lib/ee/event-destinations/dto.tsindex 95ffbab7179..06caea233fa 100644--- a/packages/core/shared/src/lib/ee/event-destinations/dto.ts+++ b/packages/core/shared/src/lib/ee/event-destinations/dto.ts@@ -6,6 +6,12 @@ export enum EventDestinationScope {PROJECT = 'PROJECT',}+export enum EventDestinationFormat {+ RAW = 'RAW',+ OTLP_JSON = 'OTLP_JSON',+ OTLP_PROTOBUF = 'OTLP_PROTOBUF',+}+export const ListPlatformEventDestinationsRequestBody = z.object({cursor: z.string().optional(),limit: z.coerce.number().optional(),@@ -13,14 +19,13 @@ export const ListPlatformEventDestinationsRequestBody = z.object({export type ListPlatformEventDestinationsRequestBody = z.infer<typeof ListPlatformEventDestinationsRequestBody>--export const CreatePlatformEventDestinationRequestBody = z.object({- events: z.array(z.nativeEnum(ApplicationEventName)),- url: z.string().url(),+ events: z.array(z.enum(ApplicationEventName)),+ url: z.url(),+ enabled: z.boolean().optional(),+ format: z.enum(EventDestinationFormat).optional(),})-export type CreatePlatformEventDestinationRequestBody = z.infer<typeof CreatePlatformEventDestinationRequestBody>export const UpdatePlatformEventDestinationRequestBody = CreatePlatformEventDestinationRequestBodydiff --git a/packages/core/shared/src/lib/ee/event-destinations/index.ts b/packages/core/shared/src/lib/ee/event-destinations/index.tsindex c191af77232..3a042a6d583 100644--- a/packages/core/shared/src/lib/ee/event-destinations/index.ts+++ b/packages/core/shared/src/lib/ee/event-destinations/index.ts@@ -1,17 +1,17 @@import { BaseModelSchema } from '@activepieces/core-utils'import { z } from 'zod'import { ApplicationEventName } from '../audit-events/index'-import { EventDestinationScope } from './dto'+import { EventDestinationFormat, EventDestinationScope } from './dto'const EventDestinationBase = {...BaseModelSchema,platformId: z.string(),- events: z.array(z.nativeEnum(ApplicationEventName)),- url: z.string().url(),+ events: z.array(z.enum(ApplicationEventName)),+ url: z.url(),+ enabled: z.boolean(),+ format: z.enum(EventDestinationFormat),}--const EventDestinationProjectScope = z.object({...EventDestinationBase,scope: z.literal(EventDestinationScope.PROJECT),diff --git a/packages/core/utils/package.json b/packages/core/utils/package.jsonindex 85e05056233..3260ab00024 100644--- a/packages/core/utils/package.json+++ b/packages/core/utils/package.json@@ -1,6 +1,6 @@{"name": "@activepieces/core-utils",- "version": "0.9.0",+ "version": "0.10.0","type": "commonjs","main": "./dist/src/index.js","scripts": {diff --git a/packages/core/utils/src/lib/activepieces-error.ts b/packages/core/utils/src/lib/activepieces-error.tsindex fa5efb87e50..c6e05ae2224 100644--- a/packages/core/utils/src/lib/activepieces-error.ts+++ b/packages/core/utils/src/lib/activepieces-error.ts@@ -98,6 +98,7 @@ export type ApErrorParams =| ExecutionStateMissingParams| GenericErrorParams| SandboxCapacityExceededParams+ | EventDestinationFormatNotSupportedByWebhookParamsexport type TriggerExecutionFailedParams = BaseErrorParams<ErrorCode.TRIGGER_EXECUTION_FAILED, {flowId: FlowId@@ -552,6 +553,10 @@ export type GenericErrorParams = BaseErrorParams<ErrorCode.GENERIC_ERROR, {export type SandboxCapacityExceededParams = BaseErrorParams<ErrorCode.SANDBOX_CAPACITY_EXCEEDED, Record<string, never>>+export type EventDestinationFormatNotSupportedByWebhookParams = BaseErrorParams<ErrorCode.EVENT_DESTINATION_FORMAT_NOT_SUPPORTED_BY_WEBHOOK, {+ format: string+}>+export enum ErrorCode {INTERNAL_SERVER_ERROR = 'INTERNAL_SERVER_ERROR',MACHINE_NOT_CONNECTED = 'MACHINE_NOT_CONNECTED',@@ -634,4 +639,5 @@ export enum ErrorCode {RESUME_LOGS_FILE_MISSING = 'RESUME_LOGS_FILE_MISSING',EXECUTION_STATE_MISSING = 'EXECUTION_STATE_MISSING',GENERIC_ERROR = 'GENERIC_ERROR',+ EVENT_DESTINATION_FORMAT_NOT_SUPPORTED_BY_WEBHOOK = 'EVENT_DESTINATION_FORMAT_NOT_SUPPORTED_BY_WEBHOOK',}diff --git a/packages/server/api/src/app/app.ts b/packages/server/api/src/app/app.tsindex 03fbccd3482..12804eb80a7 100644--- a/packages/server/api/src/app/app.ts+++ b/packages/server/api/src/app/app.ts@@ -62,6 +62,7 @@ import { adminPlatformTemplatesCloudModule } from './ee/platform/admin/templatesimport { autumnBillingProvider } from './ee/platform/platform-plan/billing-providers/autumn-billing'import { platformPlanModule } from './ee/platform/platform-plan/platform-plan.module'import { platformTeardownJobs } from './ee/platform/platform-teardown-jobs'+import { eventDestinationEntitlementHooks } from './ee/platform-webhooks/event-destination-entitlement-hooks'import { platformWebhooksModule } from './ee/platform-webhooks/platform-webhooks.module'import { projectEnterpriseHooks } from './ee/projects/ee-project-hooks'import { platformProjectBackgroundJobs } from './ee/projects/platform-project-jobs'@@ -75,6 +76,7 @@ import { scimModule } from './ee/scim/scim-module'import { secretManagersModule } from './ee/secret-managers/secret-managers.module'import { signingKeyModule } from './ee/signing-key/signing-key-module'import { userModule } from './ee/users/user.module'+import { eventDestinationHooks } from './event-destinations/event-destinations-hooks'import { fileModule } from './file/file.module'import { flagModule } from './flags/flag.module'import { flagHooks } from './flags/flags.hooks'@@ -382,6 +384,7 @@ export const setupApp = async (app: FastifyInstance): Promise<FastifyInstance> =resumePageHooks.set((log) => ({ getTheme: (params) => appearanceHelper.getTheme({ ...params, log }) }))flowPublishHooks.set((log) => ({ assertReferencesResolve: assertAgentsResolveInProject, findMissingRequiredActions: (params) => pieceSetRequiredActions(log).findMissing(params) }))aiUsageHooks.set(agentConversationCreditsHooks)+ eventDestinationHooks.set(eventDestinationEntitlementHooks)exceptionHandler.initializeSentry(system.get(AppSystemProp.SENTRY_DSN))systemJobHandlers.registerJobHandler(SystemJobName.HARD_DELETE_PLATFORM, (data) => platformTeardownJobs(app.log).hardDeletePlatformHandler(data))break@@ -422,6 +425,7 @@ export const setupApp = async (app: FastifyInstance): Promise<FastifyInstance> =resumePageHooks.set((log) => ({ getTheme: (params) => appearanceHelper.getTheme({ ...params, log }) }))flowPublishHooks.set((log) => ({ assertReferencesResolve: assertAgentsResolveInProject, findMissingRequiredActions: (params) => pieceSetRequiredActions(log).findMissing(params) }))aiUsageHooks.set(agentConversationCreditsHooks)+ eventDestinationHooks.set(eventDestinationEntitlementHooks)breakcase ApEdition.COMMUNITY:await app.register(platformProjectModule)diff --git a/packages/server/api/src/app/database/migration/postgres/1862000000000-AddEventDestinationEnabledAndFormat.ts b/packages/server/api/src/app/database/migration/postgres/1862000000000-AddEventDestinationEnabledAndFormat.tsnew file mode 100644index 00000000000..3643c8c2ada--- /dev/null+++ b/packages/server/api/src/app/database/migration/postgres/1862000000000-AddEventDestinationEnabledAndFormat.ts@@ -0,0 +1,24 @@+import { QueryRunner } from 'typeorm'+import { Migration } from '../../migration'++export class AddEventDestinationEnabledAndFormat1862000000000 implements Migration {+ name = 'AddEventDestinationEnabledAndFormat1862000000000'+ breaking = false+ release = '0.93.0'++ public async up(queryRunner: QueryRunner): Promise<void> {+ await queryRunner.query(`+ ALTER TABLE "event_destination"+ ADD COLUMN IF NOT EXISTS "enabled" boolean NOT NULL DEFAULT true,+ ADD COLUMN IF NOT EXISTS "format" character varying NOT NULL DEFAULT 'RAW'+ `)+ }++ public async down(queryRunner: QueryRunner): Promise<void> {+ await queryRunner.query(`+ ALTER TABLE "event_destination"+ DROP COLUMN IF EXISTS "enabled",+ DROP COLUMN IF EXISTS "format"+ `)+ }+}diff --git a/packages/server/api/src/app/database/postgres-connection.ts b/packages/server/api/src/app/database/postgres-connection.tsindex ec995e2fa61..b96020601d1 100644--- a/packages/server/api/src/app/database/postgres-connection.ts+++ b/packages/server/api/src/app/database/postgres-connection.ts@@ -449,6 +449,7 @@ import { AddWaitpointDeadLetteredAt1857000000000 } from './migration/postgres/18import { AddAgentFolderId1858000000000 } from './migration/postgres/1858000000000-AddAgentFolderId'import { AddPlatformModelTier1859000000000 } from './migration/postgres/1859000000000-AddPlatformModelTier'import { AddPieceSetRequiredActions1860000000000 } from './migration/postgres/1860000000000-AddPieceSetRequiredActions'+import { AddEventDestinationEnabledAndFormat1862000000000 } from './migration/postgres/1862000000000-AddEventDestinationEnabledAndFormat'const getSslConfig = (): boolean | TlsOptions => {const useSsl = system.get(AppSystemProp.POSTGRES_USE_SSL)@@ -913,6 +914,7 @@ export const getMigrations = (): (new () => Migration)[] => {AddAgentFolderId1858000000000,AddPlatformModelTier1859000000000,AddPieceSetRequiredActions1860000000000,+ AddEventDestinationEnabledAndFormat1862000000000,]return migrations}diff --git a/packages/server/api/src/app/ee/platform-webhooks/event-destination-entitlement-hooks.ts b/packages/server/api/src/app/ee/platform-webhooks/event-destination-entitlement-hooks.tsnew file mode 100644index 00000000000..7eaf8710b10--- /dev/null+++ b/packages/server/api/src/app/ee/platform-webhooks/event-destination-entitlement-hooks.ts@@ -0,0 +1,10 @@+import { FastifyBaseLogger } from 'fastify'+import { EventDestinationHooks, IsDeliveryEntitledParams } from '../../event-destinations/event-destinations-hooks'+import { platformPlanService } from '../platform/platform-plan/platform-plan.service'++export const eventDestinationEntitlementHooks = (log: FastifyBaseLogger): EventDestinationHooks => ({+ async isDeliveryEntitled({ platformId }: IsDeliveryEntitledParams): Promise<boolean> {+ const plan = await platformPlanService(log).getOrCreateForPlatform(platformId)+ return plan.eventStreamingEnabled+ },+})diff --git a/packages/server/api/src/app/ee/platform-webhooks/platform-webhooks.controller.ts b/packages/server/api/src/app/ee/platform-webhooks/platform-webhooks.controller.tsindex eaa28df0213..0becb493936 100644--- a/packages/server/api/src/app/ee/platform-webhooks/platform-webhooks.controller.ts+++ b/packages/server/api/src/app/ee/platform-webhooks/platform-webhooks.controller.ts@@ -8,7 +8,18 @@ import { eventDestinationService } from '../../event-destinations/event-destinatexport const platformWebhooksController: FastifyPluginAsyncZod = async (app) => {app.post('/', CreateEventDestinationRequest, async (req) => {- return eventDestinationService(req.log).create(req.body, req.principal.platform.id)+ return eventDestinationService(req.log).create({+ request: req.body,+ platformId: req.principal.platform.id,+ })+ })++ app.post('/:id', UpdateEventDestinationRequest, async (req) => {+ return eventDestinationService(req.log).update({+ id: req.params.id,+ platformId: req.principal.platform.id,+ request: req.body,+ })})app.patch('/:id', UpdateEventDestinationRequest, async (req) => {diff --git a/packages/server/api/src/app/ee/platform-webhooks/platform-webhooks.module.ts b/packages/server/api/src/app/ee/platform-webhooks/platform-webhooks.module.tsindex 1867dbbc077..b1a95e706cf 100644--- a/packages/server/api/src/app/ee/platform-webhooks/platform-webhooks.module.ts+++ b/packages/server/api/src/app/ee/platform-webhooks/platform-webhooks.module.ts@@ -1,9 +1,10 @@import { FastifyPluginAsync } from 'fastify'import { eventDestinationService } from '../../event-destinations/event-destinations.service'+import { platformMustHaveFeatureEnabled } from '../authentication/ee-authorization'import { platformWebhooksController } from './platform-webhooks.controller'export const platformWebhooksModule: FastifyPluginAsync = async (app) => {eventDestinationService(app.log).setup()+ app.addHook('preHandler', platformMustHaveFeatureEnabled((platform) => platform.plan.eventStreamingEnabled))await app.register(platformWebhooksController, { prefix: '/v1/event-destinations' })}-diff --git a/packages/server/api/src/app/event-destinations/event-destinations-hooks.ts b/packages/server/api/src/app/event-destinations/event-destinations-hooks.tsnew file mode 100644index 00000000000..d15eb2e8ab8--- /dev/null+++ b/packages/server/api/src/app/event-destinations/event-destinations-hooks.ts@@ -0,0 +1,16 @@+import { PlatformId } from '@activepieces/core-utils'+import { hooksFactory } from '../helper/hooks-factory'++export const eventDestinationHooks = hooksFactory.create<EventDestinationHooks>(() => ({+ async isDeliveryEntitled(): Promise<boolean> {+ return true+ },+}))++export type EventDestinationHooks = {+ isDeliveryEntitled(params: IsDeliveryEntitledParams): Promise<boolean>+}++export type IsDeliveryEntitledParams = {+ platformId: PlatformId+}diff --git a/packages/server/api/src/app/event-destinations/event-destinations.entity.ts b/packages/server/api/src/app/event-destinations/event-destinations.entity.tsindex 07840373587..969ccd1888d 100644--- a/packages/server/api/src/app/event-destinations/event-destinations.entity.ts+++ b/packages/server/api/src/app/event-destinations/event-destinations.entity.ts@@ -1,4 +1,4 @@-import { EventDestination, EventDestinationScope, Platform, Project } from '@activepieces/shared'+import { EventDestination, EventDestinationFormat, EventDestinationScope, Platform, Project } from '@activepieces/shared'import { EntitySchema } from 'typeorm'import { ApIdSchema, BaseColumnSchemaPart } from '../database/database-common'@@ -32,6 +32,16 @@ export const EventDestinationEntity = new EntitySchema<EventDestinationSchema>({type: String,nullable: false,},+ enabled: {+ type: Boolean,+ nullable: false,+ default: true,+ },+ format: {+ type: String,+ nullable: false,+ default: EventDestinationFormat.RAW,+ },},indices: [{diff --git a/packages/server/api/src/app/event-destinations/event-destinations.service.ts b/packages/server/api/src/app/event-destinations/event-destinations.service.tsindex f9d9ae7fd57..5d63ca86cf7 100644--- a/packages/server/api/src/app/event-destinations/event-destinations.service.ts+++ b/packages/server/api/src/app/event-destinations/event-destinations.service.ts@@ -1,5 +1,6 @@-import { apId, Cursor, isNil, partition, PlatformId, ProjectId, SeekPage, spreadIfDefined, tryCatch, tryCatchSync } from '@activepieces/core-utils'-import { AgentRunSource, ApplicationEvent, ApplicationEventName, buildMockEvent, CreatePlatformEventDestinationRequestBody, EventDestination, EventDestinationScope, EventPayload, FlowRunEvent, LATEST_JOB_DATA_SCHEMA_VERSION, UpdatePlatformEventDestinationRequestBody, WorkerJobType } from '@activepieces/shared'+import { ActivepiecesError, apId, Cursor, ErrorCode, isNil, partition, PlatformId, ProjectId, SeekPage, spreadIfDefined, tryCatch, tryCatchSync } from '@activepieces/core-utils'+import { OtlpExportLogsRequest, otlpLogs } from '@activepieces/server-utils'+import { AgentRunSource, ApplicationEvent, ApplicationEventName, buildMockEvent, CreatePlatformEventDestinationRequestBody, EventDestination, EventDestinationFormat, EventDestinationJobData, EventDestinationScope, EventPayload, FlowRunEvent, LATEST_JOB_DATA_SCHEMA_VERSION, UpdatePlatformEventDestinationRequestBody, WorkerJobType } from '@activepieces/shared'import { FastifyBaseLogger } from 'fastify'import { StatusCodes } from 'http-status-codes'import { ArrayContains, FindOptionsWhere } from 'typeorm'@@ -9,10 +10,13 @@ import { applicationEvents } from '../helper/application-events'import { domainHelper } from '../helper/domain-helper'import { buildPaginator } from '../helper/pagination/build-paginator'import { paginationHelper } from '../helper/pagination/pagination-utils'+import { system } from '../helper/system/system'+import { AppSystemProp } from '../helper/system/system-props'import { projectService } from '../project/project-service'import { triggerSourceService } from '../trigger/trigger-source/trigger-source-service'import { WebhookFlowVersionToRun, webhookService } from '../webhooks/webhook.service'import { jobQueue, JobType } from '../workers/job-queue/job-queue'+import { eventDestinationHooks } from './event-destinations-hooks'import {EventDestinationEntity,EventDestinationSchema,@@ -33,6 +37,8 @@ const FLOW_RUN_EVENT_ACTIONS: ReadonlySet<ApplicationEventName> = new Set([const WEBHOOK_PATH_MARKER = '/v1/webhooks/'+const PROTOBUF_CONTENT_TYPE = 'application/x-protobuf'+export const eventDestinationService = (log: FastifyBaseLogger) => ({setup(): void {applicationEvents(log).registerListeners(log, {@@ -50,10 +56,9 @@ export const eventDestinationService = (log: FastifyBaseLogger) => ({},})},- create: async (- request: CreatePlatformEventDestinationRequestBody,- platformId: string,- ): Promise<EventDestination> => {+ create: async ({ request, platformId }: CreateParams): Promise<EventDestination> => {+ const format = request.format ?? EventDestinationFormat.RAW+ assertWebhookUrlSupportsFormat({ url: request.url, format })const entity: EventDestination = {id: apId(),created: new Date().toISOString(),@@ -62,11 +67,16 @@ export const eventDestinationService = (log: FastifyBaseLogger) => ({scope: EventDestinationScope.PLATFORM,events: request.events,url: request.url,+ enabled: request.enabled ?? true,+ format,}return eventDestinationRepo().save(entity)},update: async ({ id, platformId, request }: UpdateParams): Promise<EventDestination> => {- await eventDestinationRepo().update({ id, platformId }, request)+ const existing = await eventDestinationRepo().findOneByOrFail({ id, platformId })+ const format = request.format ?? existing.format+ assertWebhookUrlSupportsFormat({ url: request.url, format })+ await eventDestinationRepo().update({ id, platformId }, { ...request, format })return eventDestinationRepo().findOneByOrFail({ id, platformId })},delete: async ({ id, platformId }: DeleteParams): Promise<void> => {@@ -106,6 +116,7 @@ export const eventDestinationService = (log: FastifyBaseLogger) => ({platformId,events: ArrayContains([event.action]),scope: EventDestinationScope.PLATFORM,+ enabled: true,}]const broadcastToProject = !isNil(projectId) && PROJECT_SCOPE_EVENTS.includes(event.action)if (broadcastToProject) {@@ -114,13 +125,26 @@ export const eventDestinationService = (log: FastifyBaseLogger) => ({projectId,events: ArrayContains([event.action]),scope: EventDestinationScope.PROJECT,+ enabled: true,})}const destinations = await eventDestinationRepo().findBy(conditions)if (destinations.length === 0) {return}- const enrichedEvent = await enrichFlowRunEvent({ event, log })+ const { data: entitled, error: entitlementError } = await tryCatch(() => eventDestinationHooks.get(log).isDeliveryEntitled({ platformId }))+ if (!isNil(entitlementError)) {+ log.warn({ error: entitlementError, platform: { id: platformId } }, 'Failed to resolve event streaming entitlement, dropping the event')+ return+ }+ if (!entitled) {+ return+ }+ const { data: enrichment, error: enrichmentError } = await tryCatch(() => enrichFlowRunEvent({ event, log }))+ if (!isNil(enrichmentError)) {+ log.warn({ error: enrichmentError, platform: { id: platformId } }, 'Failed to enrich flow run event, sending the raw event')+ }+ const enrichedEvent = enrichment ?? eventconst webhookUrlPrefix = await domainHelper.getPublicApiUrl({path: 'v1/webhooks',})@@ -131,7 +155,7 @@ export const eventDestinationService = (log: FastifyBaseLogger) => ({event: enrichedEvent,log,})- await Promise.all(destinationsToDispatch.map(({ destination, internalFlowId }) =>+ await Promise.all(destinationsToDispatch.map(async ({ destination, internalFlowId }) =>dispatchEventToDestination({log,platformId,@@ -139,7 +163,8 @@ export const eventDestinationService = (log: FastifyBaseLogger) => ({destinationId: destination.id,destinationUrl: destination.url,internalFlowId,- event: enrichedEvent,+ format: destination.format,+ body: buildDeliveryBody({ format: destination.format, event: enrichedEvent }),}),))},@@ -159,12 +184,37 @@ export const eventDestinationService = (log: FastifyBaseLogger) => ({destinationUrl: url,webhookUrlPrefix,}),- event: mockEvent,+ format: EventDestinationFormat.RAW,+ body: mockEvent,})},})+function buildDeliveryBody({ format, event }: BuildDeliveryBodyParams): DeliveryBody {+ if (format === EventDestinationFormat.RAW) {+ return event+ }+ return otlpLogs.buildExportRequest({+ event,+ environment: system.getOrThrow(AppSystemProp.ENVIRONMENT),+ })+}++function assertWebhookUrlSupportsFormat({ url, format }: AssertWebhookUrlSupportsFormatParams): void {+ const isWebhookUrl = !isNil(extractWebhookFlowIdCandidate({ destinationUrl: url }))+ if (isWebhookUrl && format === EventDestinationFormat.OTLP_PROTOBUF) {+ throw new ActivepiecesError({+ code: ErrorCode.EVENT_DESTINATION_FORMAT_NOT_SUPPORTED_BY_WEBHOOK,+ params: { format },+ })+ }+}++function deliveryJobFields({ format }: DeliveryJobFieldsParams): Pick<EventDestinationJobData, 'contentType'> {+ return format === EventDestinationFormat.OTLP_PROTOBUF ? { contentType: PROTOBUF_CONTENT_TYPE } : {}+}+const dispatchEventToDestination = async ({log,platformId,@@ -172,7 +222,8 @@ const dispatchEventToDestination = async ({destinationId,destinationUrl,internalFlowId,- event,+ format,+ body,}: DispatchEventParams): Promise<void> => {if (!isNil(internalFlowId)) {await dispatchToInternalFlow({@@ -180,7 +231,7 @@ const dispatchEventToDestination = async ({destinationId,destinationUrl,flowId: internalFlowId,- event,+ body,})return}@@ -193,7 +244,8 @@ const dispatchEventToDestination = async ({projectId,webhookId: destinationId,webhookUrl: destinationUrl,- payload: event,+ payload: body,+ ...deliveryJobFields({ format }),jobType: WorkerJobType.EVENT_DESTINATION,},})@@ -204,7 +256,7 @@ const dispatchToInternalFlow = async ({destinationId,destinationUrl,flowId,- event,+ body,}: DispatchToInternalFlowParams): Promise<void> => {const routeSuffix = webhookRouteSuffix({ destinationUrl, flowId })const isDraftOrTest = routeSuffix.startsWith('/draft') || routeSuffix === '/test'@@ -221,7 +273,7 @@ const dispatchToInternalFlow = async ({flowVersionToRun: isDraftOrTest? WebhookFlowVersionToRun.LATEST: WebhookFlowVersionToRun.LOCKED_FALL_BACK_TO_LATEST,- data: () => Promise.resolve(buildInternalWebhookPayload({ destinationUrl, event })),+ data: () => Promise.resolve(buildInternalWebhookPayload({ destinationUrl, body })),execute: routeSuffix !== '/test',failParentOnFailure: false,}))@@ -242,12 +294,12 @@ const dispatchToInternalFlow = async ({}}-const buildInternalWebhookPayload = ({ destinationUrl, event }: BuildInternalWebhookPayloadParams): EventPayload => {+const buildInternalWebhookPayload = ({ destinationUrl, body }: BuildInternalWebhookPayloadParams): EventPayload => {const { data: url } = tryCatchSync(() => new URL(destinationUrl))return {method: 'POST',headers: { 'content-type': 'application/json' },- body: event,+ body,queryParams: isNil(url) ? {} : Object.fromEntries(url.searchParams),}}@@ -346,6 +398,7 @@ const enrichFlowRunEvent = async ({ event, log }: EnrichFlowRunEventParams): Pro])return {...event,+ ...spreadIfDefined('projectDisplayName', project?.displayName ?? event.projectDisplayName),data: {...event.data,flowRun: {@@ -390,6 +443,11 @@ type DeleteParams = {platformId: string}+type CreateParams = {+ request: CreatePlatformEventDestinationRequestBody+ platformId: string+}+type UpdateParams = {id: stringplatformId: string@@ -414,6 +472,22 @@ type TestParams = {event?: ApplicationEventName}+type DeliveryBody = ApplicationEvent | OtlpExportLogsRequest++type BuildDeliveryBodyParams = {+ format: EventDestinationFormat+ event: ApplicationEvent+}++type AssertWebhookUrlSupportsFormatParams = {+ url: string+ format: EventDestinationFormat+}++type DeliveryJobFieldsParams = {+ format: EventDestinationFormat+}+type EnrichFlowRunEventParams = {event: ApplicationEventlog: FastifyBaseLogger@@ -447,7 +521,8 @@ type DispatchEventParams = {destinationId: stringdestinationUrl: stringinternalFlowId: string | null- event: ApplicationEvent+ format: EventDestinationFormat+ body: unknown}type DispatchToInternalFlowParams = {@@ -455,12 +530,12 @@ type DispatchToInternalFlowParams = {destinationId: stringdestinationUrl: stringflowId: string- event: ApplicationEvent+ body: unknown}type BuildInternalWebhookPayloadParams = {destinationUrl: string- event: ApplicationEvent+ body: unknown}type ExtractWebhookFlowIdCandidateParams = {diff --git a/packages/server/api/test/helpers/mocks/index.ts b/packages/server/api/test/helpers/mocks/index.tsindex 4505ae825dc..79f5af792d5 100644--- a/packages/server/api/test/helpers/mocks/index.ts+++ b/packages/server/api/test/helpers/mocks/index.ts@@ -1,6 +1,6 @@import { AIProviderName, apId, assertNotNullOrUndefined, ProjectRole, RoleType } from '@activepieces/core-utils'import { LATEST_CONTEXT_VERSION, PieceMetadata } from '@activepieces/pieces-framework'-import { AIProvider, ApiKey, AppConnection, AppConnectionScope, AppConnectionStatus, AppConnectionType, ApplicationEvent, ApplicationEventName, Cell, ColorName, EventDestinationScope, Field, FieldType, File, FileCompression, FileLocation, FileType, Flow, FlowOperationStatus, FlowRun, FlowRunStatus, FlowStatus, FlowTriggerType, FlowVersion, FlowVersionState, Folder, GitBranchType, GitRepo, InvitationStatus, InvitationType, KeyAlgorithm, LATEST_FLOW_SCHEMA_VERSION, OAuthApp, OtpModel, OtpState, OtpType, PackageType, PiecesFilterType, PieceType, Platform, PlatformPlan, PlatformRole, Project, ProjectIcon, ProjectMember, ProjectPlan, ProjectRelease, ProjectReleaseType, ProjectType, Record, RunEnvironment, SigningKey, Table, Template, TemplateStatus, TemplateType, User, UserIdentity, UserIdentityProvider, UserInvitation, UserStatus } from '@activepieces/shared'+import { AIProvider, ApiKey, AppConnection, AppConnectionScope, AppConnectionStatus, AppConnectionType, ApplicationEvent, ApplicationEventName, Cell, ColorName, EventDestinationFormat, EventDestinationScope, Field, FieldType, File, FileCompression, FileLocation, FileType, Flow, FlowOperationStatus, FlowRun, FlowRunStatus, FlowStatus, FlowTriggerType, FlowVersion, FlowVersionState, Folder, GitBranchType, GitRepo, InvitationStatus, InvitationType, KeyAlgorithm, LATEST_FLOW_SCHEMA_VERSION, OAuthApp, OtpModel, OtpState, OtpType, PackageType, PiecesFilterType, PieceType, Platform, PlatformPlan, PlatformRole, Project, ProjectIcon, ProjectMember, ProjectPlan, ProjectRelease, ProjectReleaseType, ProjectType, Record, RunEnvironment, SigningKey, Table, Template, TemplateStatus, TemplateType, User, UserIdentity, UserIdentityProvider, UserInvitation, UserStatus } from '@activepieces/shared'import { faker } from '@faker-js/faker'import bcrypt from 'bcrypt'import dayjs from 'dayjs'@@ -764,25 +764,7 @@ export const createMockFolder = (folder?: Partial<Folder>): Folder => {}}-export const createMockEventDestination = (eventDestination?: Partial<{- id: string- created: string- updated: string- platformId: string- projectId: string | null- events: ApplicationEventName[]- url: string- scope: EventDestinationScope-}>): {- id: string- created: string- updated: string- platformId: string- projectId: string | null- events: ApplicationEventName[]- url: string- scope: EventDestinationScope-} => {+export const createMockEventDestination = (eventDestination?: Partial<MockEventDestination>): MockEventDestination => {return {id: eventDestination?.id ?? apId(),created: eventDestination?.created ?? faker.date.recent().toISOString(),@@ -792,9 +774,24 @@ export const createMockEventDestination = (eventDestination?: Partial<{events: eventDestination?.events ?? [faker.helpers.enumValue(ApplicationEventName)],url: eventDestination?.url ?? faker.internet.url(),scope: eventDestination?.scope ?? EventDestinationScope.PLATFORM,+ enabled: eventDestination?.enabled ?? true,+ format: eventDestination?.format ?? EventDestinationFormat.RAW,}}+type MockEventDestination = {+ id: string+ created: string+ updated: string+ platformId: string+ projectId: string | null+ events: ApplicationEventName[]+ url: string+ scope: EventDestinationScope+ enabled: boolean+ format: EventDestinationFormat+}+type CreateMockPlatformWithOwnerParams = {platform?: Partial<Omit<Platform, 'ownerId'>>owner?: Partial<Omit<User, 'platformId'>>diff --git a/packages/server/api/test/integration/cloud/event-destinations/event-destination-trigger.test.ts b/packages/server/api/test/integration/cloud/event-destinations/event-destination-trigger.test.tsindex 3949aad1b25..8e9b34bafe1 100644--- a/packages/server/api/test/integration/cloud/event-destinations/event-destination-trigger.test.ts+++ b/packages/server/api/test/integration/cloud/event-destinations/event-destination-trigger.test.ts@@ -1,5 +1,5 @@import { apId } from '@activepieces/core-utils'-import { AgentActionExecutedEvent, AgentRunSource, ApplicationEventName, EventDestinationScope, FlowCreatedEvent, FlowDeletedEvent, FlowRunEvent, WorkerJobType } from '@activepieces/shared'+import { AgentActionExecutedEvent, AgentRunSource, ApplicationEventName, EventDestinationFormat, EventDestinationScope, FlowCreatedEvent, FlowDeletedEvent, FlowRunEvent, WorkerJobType } from '@activepieces/shared'import { FastifyInstance } from 'fastify'import { eventDestinationService } from '../../../../src/app/event-destinations/event-destinations.service'import { applicationEvents } from '../../../../src/app/helper/application-events'@@ -77,6 +77,8 @@ const buildFlowRunEvent = (params: { platformId: string, projectId?: string, flo}}+const ENTITLED_PLAN = { plan: { eventStreamingEnabled: true } }+let app: FastifyInstancebeforeAll(async () => {@@ -113,8 +115,59 @@ describe('Event Destination Trigger', () => {vi.restoreAllMocks()})+ it('should drop the event when the platform is not entitled to event streaming', async () => {+ const ctx = await createTestContext(app, { plan: { eventStreamingEnabled: false } })+ const destination = createMockEventDestination({+ platformId: ctx.platform.id,+ events: [ApplicationEventName.FLOW_CREATED],+ scope: EventDestinationScope.PLATFORM,+ })+ await db.save('event_destination', destination)++ await eventDestinationService(app.log).trigger({+ event: buildFlowEvent(ApplicationEventName.FLOW_CREATED, { platformId: ctx.platform.id }),+ })++ expect(addSpy).not.toHaveBeenCalled()+ expect(handleWebhookSpy).not.toHaveBeenCalled()+ })++ it('should not queue a job for a disabled destination', async () => {+ const ctx = await createTestContext(app, ENTITLED_PLAN)+ const destination = createMockEventDestination({+ platformId: ctx.platform.id,+ events: [ApplicationEventName.FLOW_CREATED],+ scope: EventDestinationScope.PLATFORM,+ enabled: false,+ })+ await db.save('event_destination', destination)++ await eventDestinationService(app.log).trigger({+ event: buildFlowEvent(ApplicationEventName.FLOW_CREATED, { platformId: ctx.platform.id }),+ })++ expect(addSpy).not.toHaveBeenCalled()+ })++ it('should not queue a job for a destination that belongs to another platform', async () => {+ const ctx = await createTestContext(app, ENTITLED_PLAN)+ const otherCtx = await createTestContext(app, ENTITLED_PLAN)+ const destination = createMockEventDestination({+ platformId: otherCtx.platform.id,+ events: [ApplicationEventName.FLOW_CREATED],+ scope: EventDestinationScope.PLATFORM,+ })+ await db.save('event_destination', destination)++ await eventDestinationService(app.log).trigger({+ event: buildFlowEvent(ApplicationEventName.FLOW_CREATED, { platformId: ctx.platform.id }),+ })++ expect(addSpy).not.toHaveBeenCalled()+ })+it('should queue job for matching PLATFORM scope destination', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const destination = createMockEventDestination({platformId: ctx.platform.id,events: [ApplicationEventName.FLOW_CREATED],@@ -141,7 +194,7 @@ describe('Event Destination Trigger', () => {})it('should NOT queue job when event action does not match', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const destination = createMockEventDestination({platformId: ctx.platform.id,events: [ApplicationEventName.FLOW_CREATED],@@ -157,8 +210,8 @@ describe('Event Destination Trigger', () => {})it('should NOT trigger destinations from a different platform', async () => {- const ctx1 = await createTestContext(app)- const ctx2 = await createTestContext(app)+ const ctx1 = await createTestContext(app, ENTITLED_PLAN)+ const ctx2 = await createTestContext(app, ENTITLED_PLAN)const destination = createMockEventDestination({platformId: ctx1.platform.id,events: [ApplicationEventName.FLOW_CREATED],@@ -174,7 +227,7 @@ describe('Event Destination Trigger', () => {})it('should trigger all matching destinations', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const dest1 = createMockEventDestination({platformId: ctx.platform.id,events: [ApplicationEventName.FLOW_CREATED],@@ -203,7 +256,7 @@ describe('Event Destination Trigger', () => {})it('should match destination when events array has multiple entries', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const destination = createMockEventDestination({platformId: ctx.platform.id,events: [ApplicationEventName.FLOW_CREATED, ApplicationEventName.FLOW_DELETED],@@ -230,7 +283,7 @@ describe('Event Destination Trigger', () => {})it('should NOT dispatch FLOW_RUN_FINISHED to a destination whose URL is the same flow webhook (recursion guard)', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const flowId = apId()const webhookUrlPrefix = await domainHelper.getPublicApiUrl({path: 'v1/webhooks',@@ -251,7 +304,7 @@ describe('Event Destination Trigger', () => {})it('should NOT dispatch FLOW_RUN_FINISHED to a destination targeting the same flow webhook on a different origin (recursion guard, embed domain)', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const flowId = apId()const destination = createMockEventDestination({platformId: ctx.platform.id,@@ -270,7 +323,7 @@ describe('Event Destination Trigger', () => {})it('should NOT dispatch FLOW_RUN_FINISHED when the self-targeting destination percent-encodes the flow id in its URL', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const flowId = apId()const encodedFlowId = `%${flowId.charCodeAt(0).toString(16).padStart(2, '0')}${flowId.slice(1)}`const destination = createMockEventDestination({@@ -290,7 +343,7 @@ describe('Event Destination Trigger', () => {})it('should still dispatch FLOW_RUN_FINISHED to a different-origin destination that targets a different flow (as an outbound job)', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const finishedFlowId = apId()const otherFlowId = apId()const destination = createMockEventDestination({@@ -315,7 +368,7 @@ describe('Event Destination Trigger', () => {})it('should skip a different-origin self-targeting destination but keep dispatching to external destinations on the same FLOW_RUN_FINISHED event', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const flowId = apId()const selfTargetingDestination = createMockEventDestination({platformId: ctx.platform.id,@@ -351,7 +404,7 @@ describe('Event Destination Trigger', () => {})it('should still dispatch FLOW_RUN_FINISHED to a same-host destination that targets a different flow (internally, without an outbound job)', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const finishedFlowId = apId()const otherFlowId = apId()const webhookUrlPrefix = await domainHelper.getPublicApiUrl({@@ -377,7 +430,7 @@ describe('Event Destination Trigger', () => {})it('should skip self-targeting destination but keep dispatching to other destinations on the same FLOW_RUN_FINISHED event', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const flowId = apId()const webhookUrlPrefix = await domainHelper.getPublicApiUrl({path: 'v1/webhooks',@@ -409,7 +462,7 @@ describe('Event Destination Trigger', () => {})it('should drop both internal destinations when two flows are mutually wired (A↔B cycle), and still fire externals', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const flowAId = apId()const flowBId = apId()const webhookUrlPrefix = await domainHelper.getPublicApiUrl({@@ -448,7 +501,7 @@ describe('Event Destination Trigger', () => {})it('should not fire any destination when an A↔B cycle has no external destinations', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const flowAId = apId()const flowBId = apId()const webhookUrlPrefix = await domainHelper.getPublicApiUrl({@@ -476,7 +529,7 @@ describe('Event Destination Trigger', () => {})it('should dispatch FLOW_RUN_FINISHED to a PROJECT scope destination matching the project', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const destination = createMockEventDestination({platformId: ctx.platform.id,projectId: ctx.project.id,@@ -500,7 +553,7 @@ describe('Event Destination Trigger', () => {})it('should NOT dispatch to a PROJECT scope destination when projectId differs', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const otherCtx = await createTestContext(app, { platform: { id: ctx.platform.id } })const destination = createMockEventDestination({platformId: ctx.platform.id,@@ -520,7 +573,7 @@ describe('Event Destination Trigger', () => {})it('should NOT fan out non-FLOW_RUN_FINISHED events to PROJECT scope destinations', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const destination = createMockEventDestination({platformId: ctx.platform.id,projectId: ctx.project.id,@@ -539,7 +592,7 @@ describe('Event Destination Trigger', () => {})it('should ship the full event (action + data + envelope) as the queued payload', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const destination = createMockEventDestination({platformId: ctx.platform.id,events: [ApplicationEventName.FLOW_CREATED],@@ -567,8 +620,64 @@ describe('Event Destination Trigger', () => {)})+ it('should queue a RAW destination with the same job fields as before formats existed', async () => {+ const ctx = await createTestContext(app, ENTITLED_PLAN)+ const destination = createMockEventDestination({+ platformId: ctx.platform.id,+ events: [ApplicationEventName.FLOW_CREATED],+ scope: EventDestinationScope.PLATFORM,+ })+ await db.save('event_destination', destination)++ const event = buildFlowEvent(ApplicationEventName.FLOW_CREATED, { platformId: ctx.platform.id })+ await eventDestinationService(app.log).trigger({ event })++ const jobData = addSpy.mock.calls[0][0].data+ expect(jobData).not.toHaveProperty('contentType')+ expect(jobData.payload).toEqual(event)+ })++ it('should queue the OTLP/JSON request without a content type for an OTLP_JSON destination', async () => {+ const ctx = await createTestContext(app, ENTITLED_PLAN)+ const destination = createMockEventDestination({+ platformId: ctx.platform.id,+ events: [ApplicationEventName.FLOW_CREATED],+ scope: EventDestinationScope.PLATFORM,+ format: EventDestinationFormat.OTLP_JSON,+ })+ await db.save('event_destination', destination)++ const event = buildFlowEvent(ApplicationEventName.FLOW_CREATED, { platformId: ctx.platform.id })+ await eventDestinationService(app.log).trigger({ event })++ const jobData = addSpy.mock.calls[0][0].data+ expect(jobData).not.toHaveProperty('contentType')+ const record = jobData.payload.resourceLogs[0].scopeLogs[0].logRecords[0]+ expect(record.eventName).toBe(ApplicationEventName.FLOW_CREATED)+ expect(JSON.parse(record.body.stringValue)).toEqual(event)+ })++ it('should mark an OTLP_PROTOBUF job with the protobuf content type and keep the JSON request as its payload', async () => {+ const ctx = await createTestContext(app, ENTITLED_PLAN)+ const destination = createMockEventDestination({+ platformId: ctx.platform.id,+ events: [ApplicationEventName.FLOW_CREATED],+ scope: EventDestinationScope.PLATFORM,+ format: EventDestinationFormat.OTLP_PROTOBUF,+ })+ await db.save('event_destination', destination)++ await eventDestinationService(app.log).trigger({+ event: buildFlowEvent(ApplicationEventName.FLOW_CREATED, { platformId: ctx.platform.id }),+ })++ const jobData = addSpy.mock.calls[0][0].data+ expect(jobData.contentType).toBe('application/x-protobuf')+ expect(jobData.payload.resourceLogs).toHaveLength(1)+ })+it('regression: ensure that we have setup the event streaming listeners', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const workerDestination = createMockEventDestination({platformId: ctx.platform.id,events: [ApplicationEventName.FLOW_RUN_FINISHED],@@ -625,7 +734,7 @@ describe('Event Destination Trigger', () => {})it('should keep every webhook-flow destination when the flow that ran an agent action is not wired as one', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const anotherFlowDestination = createMockEventDestination({platformId: ctx.platform.id,events: [ApplicationEventName.AGENT_ACTION_EXECUTED],@@ -649,7 +758,7 @@ describe('Event Destination Trigger', () => {})it('should drop both webhook-flow destinations when two flows running agent actions are mutually wired (A<->B cycle), and still fire externals', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const flowAId = apId()const flowBId = apId()const flowADestination = createMockEventDestination({@@ -683,7 +792,7 @@ describe('Event Destination Trigger', () => {})it('should drop every webhook-flow destination when an agent action cannot name the flow it ran in', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const webhookFlowDestination = createMockEventDestination({platformId: ctx.platform.id,events: [ApplicationEventName.AGENT_ACTION_EXECUTED],@@ -709,7 +818,7 @@ describe('Event Destination Trigger', () => {})it('should NOT skip a same-host destination for non flow-run events (e.g. FLOW_CREATED)', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const flowId = apId()const webhookUrlPrefix = await domainHelper.getPublicApiUrl({path: 'v1/webhooks',@@ -735,7 +844,7 @@ describe('Event Destination Trigger', () => {describe('internal same-origin dispatch (GIT-1539)', () => {it('should dispatch internally to the handler flow instead of queueing an outbound HTTP job', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const flowId = apId()const webhookUrlPrefix = await domainHelper.getPublicApiUrl({path: 'v1/webhooks',@@ -772,8 +881,33 @@ describe('Event Destination Trigger', () => {})})+ it('should send the OTLP/JSON request to an internal handler flow for the OTLP_JSON format', async () => {+ const ctx = await createTestContext(app, ENTITLED_PLAN)+ const flowId = apId()+ const webhookUrlPrefix = await domainHelper.getPublicApiUrl({+ path: 'v1/webhooks',+ })+ const destination = createMockEventDestination({+ platformId: ctx.platform.id,+ events: [ApplicationEventName.FLOW_CREATED],+ scope: EventDestinationScope.PLATFORM,+ url: `${webhookUrlPrefix}/${flowId}`,+ format: EventDestinationFormat.OTLP_JSON,+ })+ await db.save('event_destination', destination)++ await eventDestinationService(app.log).trigger({+ event: buildFlowEvent(ApplicationEventName.FLOW_CREATED, { platformId: ctx.platform.id }),+ })++ expect(addSpy).not.toHaveBeenCalled()+ const payload = await handleWebhookSpy.mock.calls[0][0].data(ctx.project.id)+ expect(payload.headers).toEqual({ 'content-type': 'application/json' })+ expect(payload.body.resourceLogs[0].scopeLogs[0].logRecords[0].eventName).toBe(ApplicationEventName.FLOW_CREATED)+ })+it('should dispatch internally when the internal URL carries a path suffix (e.g. /sync)', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const flowId = apId()const webhookUrlPrefix = await domainHelper.getPublicApiUrl({path: 'v1/webhooks',@@ -798,7 +932,7 @@ describe('Event Destination Trigger', () => {})it('should dispatch same-origin /draft and /test route URLs internally with their own version/execute semantics', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const draftFlowId = apId()const testFlowId = apId()const webhookUrlPrefix = await domainHelper.getPublicApiUrl({@@ -843,7 +977,7 @@ describe('Event Destination Trigger', () => {})it('should drop a self-targeting same-origin destination on its own flow-run event (cycle guard)', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const flowId = apId()const webhookUrlPrefix = await domainHelper.getPublicApiUrl({path: 'v1/webhooks',@@ -865,7 +999,7 @@ describe('Event Destination Trigger', () => {})it('should forward the destination URL query params to the internal handler flow', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const flowId = apId()const webhookUrlPrefix = await domainHelper.getPublicApiUrl({path: 'v1/webhooks',@@ -894,7 +1028,7 @@ describe('Event Destination Trigger', () => {})it('should NOT bypass outbound delivery for a webhook-shaped path on a different origin', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const destination = createMockEventDestination({platformId: ctx.platform.id,events: [ApplicationEventName.FLOW_CREATED],@@ -920,7 +1054,7 @@ describe('Event Destination Trigger', () => {})it('should NOT bypass outbound delivery for a same-origin URL outside the webhook path prefix', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const webhookUrlPrefix = await domainHelper.getPublicApiUrl({path: 'v1/webhooks',})@@ -948,7 +1082,7 @@ describe('Event Destination Trigger', () => {})it('should surface (not throw) when the internal handler flow rejects the event', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const flowId = apId()const webhookUrlPrefix = await domainHelper.getPublicApiUrl({path: 'v1/webhooks',@@ -971,7 +1105,7 @@ describe('Event Destination Trigger', () => {})it('test() should dispatch internally for an internal URL and queue an outbound job for an external URL', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, ENTITLED_PLAN)const flowId = apId()const webhookUrlPrefix = await domainHelper.getPublicApiUrl({path: 'v1/webhooks',diff --git a/packages/server/api/test/integration/cloud/event-destinations/event-destination.test.ts b/packages/server/api/test/integration/cloud/event-destinations/event-destination.test.tsindex 609954e59cd..42ff179fee6 100644--- a/packages/server/api/test/integration/cloud/event-destinations/event-destination.test.ts+++ b/packages/server/api/test/integration/cloud/event-destinations/event-destination.test.ts@@ -1,14 +1,19 @@-import { apId } from '@activepieces/core-utils'-import { ApplicationEventName, PlatformRole, PrincipalType } from '@activepieces/shared'+import { apId, ErrorCode } from '@activepieces/core-utils'+import { ApplicationEventName, EventDestinationFormat, PlatformRole, PrincipalType } from '@activepieces/shared'import { FastifyInstance } from 'fastify'import { StatusCodes } from 'http-status-codes'+import { domainHelper } from '../../../../src/app/helper/domain-helper'import { generateMockToken } from '../../../helpers/auth'import { mockBasicUser } from '../../../helpers/mocks'-import { createTestContext } from '../../../helpers/test-context'+import { createTestContext, TestContext } from '../../../helpers/test-context'import { setupTestEnvironment, teardownTestEnvironment } from '../../../helpers/test-setup'let app: FastifyInstance | null = null+const createEnabledContext = async (): Promise<TestContext> => createTestContext(app!, {+ plan: { eventStreamingEnabled: true },+})+beforeAll(async () => {app = await setupTestEnvironment()})@@ -20,7 +25,7 @@ afterAll(async () => {describe('Event Destinations API', () => {describe('POST /v1/event-destinations (Create)', () => {it('should create an event destination', async () => {- const ctx = await createTestContext(app!)+ const ctx = await createEnabledContext()const response = await ctx.post('/v1/event-destinations', {url: 'https://example.com/webhook',@@ -34,11 +39,80 @@ describe('Event Destinations API', () => {expect(body.platformId).toBe(ctx.platform.id)expect(body.id).toBeDefined()})++ it('should default the format to RAW when the request does not send one', async () => {+ const ctx = await createEnabledContext()++ const response = await ctx.post('/v1/event-destinations', {+ url: 'https://example.com/webhook',+ events: [ApplicationEventName.FLOW_CREATED],+ })++ expect(response?.statusCode).toBe(StatusCodes.OK)+ expect(response?.json().format).toBe(EventDestinationFormat.RAW)+ })++ it('should store and return the requested format', async () => {+ const ctx = await createEnabledContext()++ const created = await ctx.post('/v1/event-destinations', {+ url: 'https://otlp.example.com/v1/logs',+ events: [ApplicationEventName.FLOW_CREATED],+ format: EventDestinationFormat.OTLP_PROTOBUF,+ })+ const listed = await ctx.get('/v1/event-destinations')++ expect(created?.json().format).toBe(EventDestinationFormat.OTLP_PROTOBUF)+ expect(listed?.json().data[0].format).toBe(EventDestinationFormat.OTLP_PROTOBUF)+ })++ it('should refuse OTLP_PROTOBUF for a webhook URL on this instance, because a webhook accepts only JSON', async () => {+ const ctx = await createEnabledContext()+ const webhookUrlPrefix = await domainHelper.getPublicApiUrl({ path: 'v1/webhooks' })++ const response = await ctx.post('/v1/event-destinations', {+ url: `${webhookUrlPrefix}/${apId()}`,+ events: [ApplicationEventName.FLOW_CREATED],+ format: EventDestinationFormat.OTLP_PROTOBUF,+ })+ const listed = await ctx.get('/v1/event-destinations')++ expect(response?.statusCode).toBe(StatusCodes.BAD_REQUEST)+ expect(response?.json().code).toBe(ErrorCode.EVENT_DESTINATION_FORMAT_NOT_SUPPORTED_BY_WEBHOOK)+ expect(listed?.json().data).toHaveLength(0)+ })++ it('should refuse OTLP_PROTOBUF for a webhook URL on another host, such as an embed subdomain', async () => {+ const ctx = await createEnabledContext()++ const response = await ctx.post('/v1/event-destinations', {+ url: `https://automations.customer.example/api/v1/webhooks/${apId()}/sync`,+ events: [ApplicationEventName.FLOW_CREATED],+ format: EventDestinationFormat.OTLP_PROTOBUF,+ })++ expect(response?.statusCode).toBe(StatusCodes.BAD_REQUEST)+ expect(response?.json().code).toBe(ErrorCode.EVENT_DESTINATION_FORMAT_NOT_SUPPORTED_BY_WEBHOOK)+ })++ it('should accept OTLP_JSON for a webhook URL, because it arrives as JSON', async () => {+ const ctx = await createEnabledContext()+ const webhookUrlPrefix = await domainHelper.getPublicApiUrl({ path: 'v1/webhooks' })++ const response = await ctx.post('/v1/event-destinations', {+ url: `${webhookUrlPrefix}/${apId()}`,+ events: [ApplicationEventName.FLOW_CREATED],+ format: EventDestinationFormat.OTLP_JSON,+ })++ expect(response?.statusCode).toBe(StatusCodes.OK)+ expect(response?.json().format).toBe(EventDestinationFormat.OTLP_JSON)+ })})describe('GET /v1/event-destinations (List)', () => {it('should list event destinations', async () => {- const ctx = await createTestContext(app!)+ const ctx = await createEnabledContext()await ctx.post('/v1/event-destinations', {url: 'https://example.com/webhook1',@@ -53,7 +127,7 @@ describe('Event Destinations API', () => {})it('should return empty list for new platform', async () => {- const ctx = await createTestContext(app!)+ const ctx = await createEnabledContext()const response = await ctx.get('/v1/event-destinations')@@ -64,9 +138,9 @@ describe('Event Destinations API', () => {})})- describe('PATCH /v1/event-destinations/:id (Update)', () => {+ describe('POST /v1/event-destinations/:id (Update)', () => {it('should update event destination', async () => {- const ctx = await createTestContext(app!)+ const ctx = await createEnabledContext()const createResponse = await ctx.post('/v1/event-destinations', {url: 'https://example.com/original',@@ -75,7 +149,7 @@ describe('Event Destinations API', () => {const destId = createResponse?.json().idconst response = await ctx.inject({- method: 'PATCH',+ method: 'POST',url: `/api/v1/event-destinations/${destId}`,body: {url: 'https://example.com/updated',@@ -89,12 +163,92 @@ describe('Event Destinations API', () => {expect(body.events).toContain(ApplicationEventName.FLOW_DELETED)})+ it('should keep the stored format when the update does not send one', async () => {+ const ctx = await createEnabledContext()+ const created = await ctx.post('/v1/event-destinations', {+ url: 'https://otlp.example.com/v1/logs',+ events: [ApplicationEventName.FLOW_CREATED],+ format: EventDestinationFormat.OTLP_JSON,+ })++ const response = await ctx.post(`/v1/event-destinations/${created?.json().id}`, {+ url: 'https://otlp.example.com/v1/logs',+ events: [ApplicationEventName.FLOW_DELETED],+ })++ expect(response?.statusCode).toBe(StatusCodes.OK)+ expect(response?.json().format).toBe(EventDestinationFormat.OTLP_JSON)+ })++ it('should refuse an update that sets OTLP_PROTOBUF on a webhook URL, and keep the stored format', async () => {+ const ctx = await createEnabledContext()+ const webhookUrl = `${await domainHelper.getPublicApiUrl({ path: 'v1/webhooks' })}/${apId()}`+ const created = await ctx.post('/v1/event-destinations', {+ url: webhookUrl,+ events: [ApplicationEventName.FLOW_CREATED],+ })++ const response = await ctx.post(`/v1/event-destinations/${created?.json().id}`, {+ url: webhookUrl,+ events: [ApplicationEventName.FLOW_CREATED],+ format: EventDestinationFormat.OTLP_PROTOBUF,+ })+ const listed = await ctx.get('/v1/event-destinations')++ expect(response?.statusCode).toBe(StatusCodes.BAD_REQUEST)+ expect(response?.json().code).toBe(ErrorCode.EVENT_DESTINATION_FORMAT_NOT_SUPPORTED_BY_WEBHOOK)+ expect(listed?.json().data[0].format).toBe(EventDestinationFormat.RAW)+ })++ it('should refuse an update that moves an OTLP_PROTOBUF destination to a webhook URL, and keep the stored URL', async () => {+ const ctx = await createEnabledContext()+ const created = await ctx.post('/v1/event-destinations', {+ url: 'https://otlp.example.com/v1/logs',+ events: [ApplicationEventName.FLOW_CREATED],+ format: EventDestinationFormat.OTLP_PROTOBUF,+ })++ const response = await ctx.inject({+ method: 'PATCH',+ url: `/api/v1/event-destinations/${created?.json().id}`,+ body: {+ url: `https://automations.customer.example/api/v1/webhooks/${apId()}`,+ events: [ApplicationEventName.FLOW_CREATED],+ },+ })+ const listed = await ctx.get('/v1/event-destinations')++ expect(response?.statusCode).toBe(StatusCodes.BAD_REQUEST)+ expect(response?.json().code).toBe(ErrorCode.EVENT_DESTINATION_FORMAT_NOT_SUPPORTED_BY_WEBHOOK)+ expect(listed?.json().data[0].url).toBe('https://otlp.example.com/v1/logs')+ })++ it('should still accept PATCH, the method existing API clients use', async () => {+ const ctx = await createEnabledContext()+ const created = await ctx.post('/v1/event-destinations', {+ url: 'https://example.com/original',+ events: [ApplicationEventName.FLOW_CREATED],+ })++ const response = await ctx.inject({+ method: 'PATCH',+ url: `/api/v1/event-destinations/${created?.json().id}`,+ body: {+ url: 'https://example.com/patched',+ events: [ApplicationEventName.FLOW_CREATED],+ },+ })++ expect(response?.statusCode).toBe(StatusCodes.OK)+ expect(response?.json().url).toBe('https://example.com/patched')+ })+it('should return error for non-existent destination', async () => {- const ctx = await createTestContext(app!)+ const ctx = await createEnabledContext()const nonExistentId = apId()const response = await ctx.inject({- method: 'PATCH',+ method: 'POST',url: `/api/v1/event-destinations/${nonExistentId}`,body: {url: 'https://example.com/updated',@@ -109,7 +263,7 @@ describe('Event Destinations API', () => {describe('DELETE /v1/event-destinations/:id', () => {it('should delete an event destination', async () => {- const ctx = await createTestContext(app!)+ const ctx = await createEnabledContext()const createResponse = await ctx.post('/v1/event-destinations', {url: 'https://example.com/delete-me',@@ -123,7 +277,7 @@ describe('Event Destinations API', () => {})it('should return 200 for non-existent destination (idempotent delete)', async () => {- const ctx = await createTestContext(app!)+ const ctx = await createEnabledContext()const nonExistentId = apId()const response = await ctx.delete(`/v1/event-destinations/${nonExistentId}`)@@ -134,7 +288,7 @@ describe('Event Destinations API', () => {describe('POST /v1/event-destinations/test', () => {it('should accept a test request with a webhook URL and an event name', async () => {- const ctx = await createTestContext(app!)+ const ctx = await createEnabledContext()const response = await ctx.post('/v1/event-destinations/test', {url: 'https://example.com/webhook',@@ -145,7 +299,7 @@ describe('Event Destinations API', () => {})it('should accept a test request with no event (defaults to flow.created)', async () => {- const ctx = await createTestContext(app!)+ const ctx = await createEnabledContext()const response = await ctx.post('/v1/event-destinations/test', {url: 'https://example.com/webhook',@@ -156,8 +310,23 @@ describe('Event Destinations API', () => {})describe('Auth', () => {+ it('should return 402 when the platform plan has event streaming disabled', async () => {+ const ctx = await createTestContext(app!, {+ plan: { eventStreamingEnabled: false },+ })++ const created = await ctx.post('/v1/event-destinations', {+ url: 'https://example.com/webhook',+ events: [ApplicationEventName.FLOW_CREATED],+ })+ expect(created?.statusCode).toBe(StatusCodes.PAYMENT_REQUIRED)++ const listed = await ctx.get('/v1/event-destinations')+ expect(listed?.statusCode).toBe(StatusCodes.PAYMENT_REQUIRED)+ })+it('should return 403 for non-admin user', async () => {- const ctx = await createTestContext(app!)+ const ctx = await createEnabledContext()const { mockUser } = await mockBasicUser({user: {@@ -186,8 +355,8 @@ describe('Event Destinations API', () => {})it('should isolate event destinations between platforms', async () => {- const ctx1 = await createTestContext(app!)- const ctx2 = await createTestContext(app!)+ const ctx1 = await createEnabledContext()+ const ctx2 = await createEnabledContext()await ctx1.post('/v1/event-destinations', {url: 'https://example.com/platform1',diff --git a/packages/server/api/test/integration/cloud/event-destinations/mcp-flow-published-delivery.test.ts b/packages/server/api/test/integration/cloud/event-destinations/mcp-flow-published-delivery.test.tsindex 6e2ecc4a2f0..6ed4e8d8e2a 100644--- a/packages/server/api/test/integration/cloud/event-destinations/mcp-flow-published-delivery.test.ts+++ b/packages/server/api/test/integration/cloud/event-destinations/mcp-flow-published-delivery.test.ts@@ -47,7 +47,7 @@ describe('MCP publish delivers flow.published to event destinations', () => {})it('queues an EVENT_DESTINATION job when publishing through ap_lock_and_publish', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, { plan: { eventStreamingEnabled: true } })await db.save('event_destination', createMockEventDestination({platformId: ctx.platform.id,events: [ApplicationEventName.FLOW_PUBLISHED],@@ -67,7 +67,7 @@ describe('MCP publish delivers flow.published to event destinations', () => {})it('queues no job when the destination does not subscribe to flow.published', async () => {- const ctx = await createTestContext(app)+ const ctx = await createTestContext(app, { plan: { eventStreamingEnabled: true } })await db.save('event_destination', createMockEventDestination({platformId: ctx.platform.id,events: [ApplicationEventName.FLOW_RUN_FINISHED],diff --git a/packages/server/utils/package.json b/packages/server/utils/package.jsonindex 79c149ec9b8..160b35c8e64 100644--- a/packages/server/utils/package.json+++ b/packages/server/utils/package.json@@ -13,6 +13,7 @@"@activepieces/shared": "workspace:*","@modelcontextprotocol/sdk": "1.27.1","evlog": "2.22.4",+ "protobufjs": "7.6.6","@ai-sdk/amazon-bedrock": "5.0.38","@ai-sdk/anthropic": "4.0.25","@ai-sdk/azure": "4.0.26",diff --git a/packages/server/utils/src/index.ts b/packages/server/utils/src/index.tsindex 35df74803bc..945478e43dc 100644--- a/packages/server/utils/src/index.ts+++ b/packages/server/utils/src/index.ts@@ -5,6 +5,8 @@ export { evlogDrains } from './evlog-drains'export type { EvlogDrainConfig } from './evlog-drains'export { evlogSetup } from './evlog-setup'export type { EvlogSetupParams } from './evlog-setup'+export { otlpLogs } from './otlp-logs'+export type { OtlpExportLogsRequest, OtlpLogRecord } from './otlp-logs'export { createLogger, defineAuditAction, defineErrorCatalog, parseError } from 'evlog'export type { AuditActor, DefinedAuditAction, ErrorCatalog } from 'evlog'export { evlog as evlogFastify, useLogger as useWideEventLogger } from 'evlog/fastify'@@ -25,8 +27,8 @@ export { modelCatalog } from './model-catalog'export { modelTierCatalog } from './model-tier-catalog'export type { ModelTier, ModelTierReader, ModelTierSurface } from './model-tier-catalog'export { RedisType } from './redis-type'-export { safeHttp } from './safe-http'-export type { SsrfAgents } from './safe-http'+export { PostForStatusFailure, safeHttp } from './safe-http'+export type { PostForStatusResult, SsrfAgents } from './safe-http'export { DEFAULT_MCP_MAX_RESPONSE_BYTES, DEFAULT_MCP_TIMEOUT_MS, mcpTransport } from './mcp-transport'export { systemUsage } from './system-usage'export { activepiecesAiCost } from './activepieces-ai-cost'diff --git a/packages/server/utils/src/otlp-logs.ts b/packages/server/utils/src/otlp-logs.tsnew file mode 100644index 00000000000..dcb969cf3bb--- /dev/null+++ b/packages/server/utils/src/otlp-logs.ts@@ -0,0 +1,207 @@+import { isObject } from '@activepieces/core-utils'+import { ApplicationEvent } from '@activepieces/shared'+import { OTLPLogRecord, toOTLPLogRecord } from 'evlog/otlp'+import { Root } from 'protobufjs'++const SERVICE_NAME = 'activepieces'++const SCOPE_NAME = 'activepieces.event-streaming'++const KEY_VALUE = {+ fields: {+ key: { type: 'string', id: 1 },+ value: { type: 'AnyValue', id: 2 },+ },+}++const OTLP_LOGS_DESCRIPTOR = {+ nested: {+ otlp: {+ nested: {+ AnyValue: {+ oneofs: {+ value: {+ oneof: ['stringValue', 'boolValue', 'intValue', 'doubleValue', 'arrayValue', 'kvlistValue', 'bytesValue'],+ },+ },+ fields: {+ stringValue: { type: 'string', id: 1 },+ boolValue: { type: 'bool', id: 2 },+ intValue: { type: 'int64', id: 3 },+ doubleValue: { type: 'double', id: 4 },+ arrayValue: { type: 'ArrayValue', id: 5 },+ kvlistValue: { type: 'KeyValueList', id: 6 },+ bytesValue: { type: 'bytes', id: 7 },+ },+ },+ ArrayValue: {+ fields: {+ values: { rule: 'repeated', type: 'AnyValue', id: 1 },+ },+ },+ KeyValueList: {+ fields: {+ values: { rule: 'repeated', type: 'KeyValue', id: 1 },+ },+ },+ KeyValue: KEY_VALUE,+ Resource: {+ fields: {+ attributes: { rule: 'repeated', type: 'KeyValue', id: 1 },+ droppedAttributesCount: { type: 'uint32', id: 2 },+ },+ },+ InstrumentationScope: {+ fields: {+ name: { type: 'string', id: 1 },+ version: { type: 'string', id: 2 },+ attributes: { rule: 'repeated', type: 'KeyValue', id: 3 },+ droppedAttributesCount: { type: 'uint32', id: 4 },+ },+ },+ LogRecord: {+ fields: {+ timeUnixNano: { type: 'fixed64', id: 1 },+ severityNumber: { type: 'int32', id: 2 },+ severityText: { type: 'string', id: 3 },+ body: { type: 'AnyValue', id: 5 },+ attributes: { rule: 'repeated', type: 'KeyValue', id: 6 },+ droppedAttributesCount: { type: 'uint32', id: 7 },+ flags: { type: 'fixed32', id: 8 },+ traceId: { type: 'bytes', id: 9 },+ spanId: { type: 'bytes', id: 10 },+ observedTimeUnixNano: { type: 'fixed64', id: 11 },+ eventName: { type: 'string', id: 12 },+ },+ },+ ScopeLogs: {+ fields: {+ scope: { type: 'InstrumentationScope', id: 1 },+ logRecords: { rule: 'repeated', type: 'LogRecord', id: 2 },+ schemaUrl: { type: 'string', id: 3 },+ },+ },+ ResourceLogs: {+ fields: {+ resource: { type: 'Resource', id: 1 },+ scopeLogs: { rule: 'repeated', type: 'ScopeLogs', id: 2 },+ schemaUrl: { type: 'string', id: 3 },+ },+ },+ ExportLogsServiceRequest: {+ fields: {+ resourceLogs: { rule: 'repeated', type: 'ResourceLogs', id: 1 },+ },+ },+ },+ },+ },+}++const exportLogsServiceRequest = Root.fromJSON(OTLP_LOGS_DESCRIPTOR).lookupType('otlp.ExportLogsServiceRequest')++function buildExportRequest({ event, environment }: BuildExportRequestParams): OtlpExportLogsRequest {+ const { created, data, action, id, platformId, projectId, projectDisplayName, userId, userEmail, ip } = event+ const dataAttributes: Record<string, unknown> = Object.fromEntries(flattenAttributes({ prefix: 'data', value: data }))+ const record = toOTLPLogRecord({+ timestamp: created,+ level: 'info',+ service: SERVICE_NAME,+ environment,+ action,+ id,+ platformId,+ projectId,+ projectDisplayName,+ userId,+ userEmail,+ ip,+ ...dataAttributes,+ })+ return {+ resourceLogs: [{+ resource: {+ attributes: [{ key: 'service.name', value: { stringValue: SERVICE_NAME } }],+ },+ scopeLogs: [{+ scope: { name: SCOPE_NAME },+ logRecords: [{+ ...record,+ observedTimeUnixNano: record.timeUnixNano,+ attributes: record.attributes.map((attribute) => withDoubleValue({ attribute, value: dataAttributes[attribute.key] })),+ body: { stringValue: JSON.stringify(event) },+ eventName: action,+ }],+ }],+ }],+ }+}++function encodeExportRequest(request: Record<string, unknown>): Uint8Array {+ return exportLogsServiceRequest.encode(exportLogsServiceRequest.fromObject(request)).finish()+}++export const otlpLogs = {+ buildExportRequest,+ encodeExportRequest,+}++function flattenAttributes({ prefix, value }: FlattenAttributesParams): [string, unknown][] {+ if (value instanceof Date) {+ return [[prefix, value.toISOString()]]+ }+ if (isObject(value)) {+ return Object.entries(value).flatMap(([key, child]) => flattenAttributes({ prefix: `${prefix}.${key}`, value: child }))+ }+ return [[prefix, value]]+}++function withDoubleValue({ attribute, value }: WithDoubleValueParams): OtlpAttribute {+ if (typeof value !== 'number' || !Number.isFinite(value) || Number.isInteger(value)) {+ return attribute+ }+ return { key: attribute.key, value: { doubleValue: value } }+}++type BuildExportRequestParams = {+ event: ApplicationEvent+ environment: string+}++type FlattenAttributesParams = {+ prefix: string+ value: unknown+}++type WithDoubleValueParams = {+ attribute: OtlpAttribute+ value: unknown+}++type OtlpAttribute = {+ key: string+ value: {+ stringValue?: string+ intValue?: string+ boolValue?: boolean+ doubleValue?: number+ }+}++export type OtlpLogRecord = Omit<OTLPLogRecord, 'attributes'> & {+ attributes: OtlpAttribute[]+ observedTimeUnixNano: string+ eventName: string+}++export type OtlpExportLogsRequest = {+ resourceLogs: {+ resource: {+ attributes: OtlpAttribute[]+ }+ scopeLogs: {+ scope: { name: string }+ logRecords: OtlpLogRecord[]+ }[]+ }[]+}diff --git a/packages/server/utils/src/safe-http.ts b/packages/server/utils/src/safe-http.tsindex 2bf6f6a46fa..4319a4e162c 100644--- a/packages/server/utils/src/safe-http.ts+++ b/packages/server/utils/src/safe-http.ts@@ -1,6 +1,7 @@import http from 'node:http'import https from 'node:https'-import { isNil } from '@activepieces/core-utils';+import { finished, Readable } from 'node:stream'+import { isNil, tryCatch } from '@activepieces/core-utils';import axios, { AxiosError, AxiosInstance, AxiosRequestConfig } from 'axios'import axiosRetry from 'axios-retry'import { RequestFilteringHttpAgent, RequestFilteringHttpsAgent } from 'request-filtering-agent'@@ -67,16 +68,84 @@ function createRetryingAxios(config?: AxiosRequestConfig, options?: SafeAxiosOptreturn instance}+async function postForStatus({ url, headers, body, timeoutMs }: PostForStatusParams): Promise<PostForStatusResult> {+ const signal = AbortSignal.timeout(timeoutMs)+ const { data: response, error } = await tryCatch(() => safeHttp.axios.request<Readable>({+ url,+ method: 'POST',+ headers,+ data: body,+ signal,+ responseType: 'stream',+ decompress: false,+ validateStatus: () => true,+ }))+ if (error !== null) {+ return { responded: false, failure: toPostFailure({ error, signal }), error }+ }+ await discardResponseBody({ body: response.data, signal })+ return { responded: true, status: response.status }+}++function discardResponseBody({ body, signal }: DiscardResponseBodyParams): Promise<void> {+ return new Promise((resolve) => {+ const stop = (): void => {+ body.destroy()+ }+ let receivedBytes = 0+ body.on('data', (chunk: Buffer) => {+ receivedBytes += chunk.length+ if (receivedBytes > MAX_DRAINED_RESPONSE_BYTES) {+ stop()+ }+ })+ finished(body, () => {+ signal.removeEventListener('abort', stop)+ resolve()+ })+ if (signal.aborted) {+ stop()+ return+ }+ signal.addEventListener('abort', stop, { once: true })+ })+}++function toPostFailure({ error, signal }: ToPostFailureParams): PostForStatusFailure {+ const code = errorCodeOf(error)+ if (signal.aborted || TIMEOUT_ERROR_CODES.includes(code)) {+ return PostForStatusFailure.TIMEOUT+ }+ if (isSsrfFilterError(error)) {+ return PostForStatusFailure.BLOCKED+ }+ if (TLS_ERROR_CODE_REGEX.test(code)) {+ return PostForStatusFailure.TLS+ }+ return PostForStatusFailure.CONNECTION_FAILED+}++function errorCodeOf(error: Error): string {+ if ('code' in error && typeof error.code === 'string') {+ return error.code+ }+ return error.cause instanceof Error ? errorCodeOf(error.cause) : ''+}+let lazyDefaultAxios: AxiosInstance | undefinedlet lazyRetryingAxios: AxiosInstance | undefinedconst SSRF_FILTER_MESSAGE_REGEX = /(DNS lookup .* not allowed|IP .* is not allowed)/iconst SSRF_REMEDIATION_HINT = 'the target is blocked by the SSRF filter. If it is a trusted internal host (e.g. a self-hosted Vault, Conjur, or OAuth2 provider), add its IP or CIDR to the AP_SSRF_ALLOW_LIST environment variable (comma-separated) and restart the server.'+const MAX_DRAINED_RESPONSE_BYTES = 16 * 1024+const TIMEOUT_ERROR_CODES = ['ETIMEDOUT', 'ECONNABORTED']+const TLS_ERROR_CODE_REGEX = /^(ERR_TLS_|ERR_SSL_|CERT_|UNABLE_TO_|DEPTH_ZERO_SELF_SIGNED_CERT$|SELF_SIGNED_CERT_IN_CHAIN$|EPROTO$)/export const safeHttp = {buildAgents,createAxios,createRetryingAxios,+ postForStatus,get axios(): AxiosInstance {lazyDefaultAxios ??= createAxios()return lazyDefaultAxios@@ -87,11 +156,39 @@ export const safeHttp = {},}+export enum PostForStatusFailure {+ BLOCKED = 'BLOCKED',+ TIMEOUT = 'TIMEOUT',+ TLS = 'TLS',+ CONNECTION_FAILED = 'CONNECTION_FAILED',+}++export type PostForStatusResult =+ | { responded: true, status: number }+ | { responded: false, failure: PostForStatusFailure, error: Error }+export type SsrfAgents = {httpAgent: http.AgenthttpsAgent: https.Agent}+type PostForStatusParams = {+ url: string+ headers: Record<string, string>+ body: unknown+ timeoutMs: number+}++type DiscardResponseBodyParams = {+ body: Readable+ signal: AbortSignal+}++type ToPostFailureParams = {+ error: Error+ signal: AbortSignal+}+export type SafeAxiosOptions = {httpsAgentOptions?: https.AgentOptions}diff --git a/packages/server/utils/test/otlp-logs.test.ts b/packages/server/utils/test/otlp-logs.test.tsnew file mode 100644index 00000000000..76e9984a53f--- /dev/null+++ b/packages/server/utils/test/otlp-logs.test.ts@@ -0,0 +1,188 @@+import { ApplicationEventName, buildMockEvent, ConnectionEvent, FlowRunEvent } from '@activepieces/shared'+import { Root } from 'protobufjs'+import { describe, expect, it } from 'vitest'+import { otlpLogs } from '../src/otlp-logs'++const flowRunFinished: FlowRunEvent = {+ id: 'Qz3vN8kLp2XwR7tY1mB4c',+ created: '2026-09-23T10:15:42.318Z',+ updated: '2026-09-23T10:15:42.318Z',+ platformId: 'Hd8sK2mWq9LxT4vB7nP1e',+ projectId: 'Rt6yU1oPa3SdF5gH8jK2l',+ projectDisplayName: 'Finance Ops',+ action: ApplicationEventName.FLOW_RUN_FINISHED,+ data: {+ flowRun: {+ id: 'Mn2bV5cX8zL1kJ4hG7fD0',+ startTime: '2026-09-23T10:15:37.087Z',+ finishTime: '2026-09-23T10:15:42.301Z',+ duration: 5214,+ environment: 'PRODUCTION',+ flowId: 'Wq4eR7tY0uI3oP6aS9dF2',+ flowVersionId: 'Zx1cV4bN7mL0kJ3hG6fD9',+ flowDisplayName: 'Invoice sync',+ status: 'FAILED',+ },+ project: { displayName: 'Finance Ops' },+ },+}++describe('otlpLogs.buildExportRequest', () => {+ it('pins the OTLP logs request shape for a flow run event', () => {+ const request = otlpLogs.buildExportRequest({ event: flowRunFinished, environment: 'prod' })++ expect(request).toEqual({+ resourceLogs: [{+ resource: { attributes: [{ key: 'service.name', value: { stringValue: 'activepieces' } }] },+ scopeLogs: [{+ scope: { name: 'activepieces.event-streaming' },+ logRecords: [{+ timeUnixNano: '1790158542318000000',+ observedTimeUnixNano: '1790158542318000000',+ severityNumber: 9,+ severityText: 'INFO',+ eventName: 'flow.run.finished',+ body: { stringValue: JSON.stringify(flowRunFinished) },+ attributes: [+ { key: 'action', value: { stringValue: 'flow.run.finished' } },+ { key: 'id', value: { stringValue: 'Qz3vN8kLp2XwR7tY1mB4c' } },+ { key: 'platformId', value: { stringValue: 'Hd8sK2mWq9LxT4vB7nP1e' } },+ { key: 'projectId', value: { stringValue: 'Rt6yU1oPa3SdF5gH8jK2l' } },+ { key: 'projectDisplayName', value: { stringValue: 'Finance Ops' } },+ { key: 'data.flowRun.id', value: { stringValue: 'Mn2bV5cX8zL1kJ4hG7fD0' } },+ { key: 'data.flowRun.startTime', value: { stringValue: '2026-09-23T10:15:37.087Z' } },+ { key: 'data.flowRun.finishTime', value: { stringValue: '2026-09-23T10:15:42.301Z' } },+ { key: 'data.flowRun.duration', value: { intValue: '5214' } },+ { key: 'data.flowRun.environment', value: { stringValue: 'PRODUCTION' } },+ { key: 'data.flowRun.flowId', value: { stringValue: 'Wq4eR7tY0uI3oP6aS9dF2' } },+ { key: 'data.flowRun.flowVersionId', value: { stringValue: 'Zx1cV4bN7mL0kJ3hG6fD9' } },+ { key: 'data.flowRun.flowDisplayName', value: { stringValue: 'Invoice sync' } },+ { key: 'data.flowRun.status', value: { stringValue: 'FAILED' } },+ { key: 'data.project.displayName', value: { stringValue: 'Finance Ops' } },+ ],+ }],+ }],+ }],+ })+ })++ it('leaves out the envelope fields a worker-sourced event does not carry', () => {+ const keys = attributeKeysOf(otlpLogs.buildExportRequest({ event: flowRunFinished, environment: 'prod' }))++ expect(keys).not.toContain('userId')+ expect(keys).not.toContain('userEmail')+ expect(keys).not.toContain('ip')+ expect(keys).not.toContain('created')+ expect(keys).not.toContain('updated')+ expect(keys).not.toContain('data')+ expect(keys).not.toContain('environment')+ })++ it('keeps the user fields of a request-sourced event', () => {+ const event = buildMockEvent({ event: ApplicationEventName.FLOW_CREATED, platformId: 'platform-1', projectId: 'project-1' })+ const keys = attributeKeysOf(otlpLogs.buildExportRequest({ event, environment: 'prod' }))++ expect(keys).toEqual(expect.arrayContaining(['userId', 'userEmail', 'ip', 'projectDisplayName']))+ })++ it('sends an array as one JSON string and a date as an ISO string', () => {+ const piecesUpgraded = buildMockEvent({ event: ApplicationEventName.FLOW_PIECES_UPGRADED, platformId: 'platform-1', projectId: 'project-1' })+ const connection: ConnectionEvent = {+ ...buildMockEvent({ event: ApplicationEventName.CONNECTION_UPSERTED, platformId: 'platform-1', projectId: 'project-1' }),+ action: ApplicationEventName.CONNECTION_UPSERTED,+ data: {+ connection: {+ id: 'connection-1',+ displayName: 'Sample connection',+ externalId: 'sample-connection',+ pieceName: '@activepieces/piece-sample',+ status: 'ACTIVE',+ type: 'CUSTOM_AUTH',+ created: new Date('2026-09-23T10:15:42.318Z'),+ updated: '2026-09-23T10:15:42.318Z',+ },+ },+ }++ const steps = attributeOf({ request: otlpLogs.buildExportRequest({ event: piecesUpgraded, environment: 'prod' }), key: 'data.steps' })+ const created = attributeOf({ request: otlpLogs.buildExportRequest({ event: connection, environment: 'prod' }), key: 'data.connection.created' })++ expect(steps?.value.stringValue).toBe(JSON.stringify('steps' in piecesUpgraded.data ? piecesUpgraded.data.steps : undefined))+ expect(created?.value.stringValue).toBe('2026-09-23T10:15:42.318Z')+ })++ it('sends a decimal number as a double and a whole number as an integer', () => {+ const event: FlowRunEvent = { ...flowRunFinished, data: { ...flowRunFinished.data, flowRun: { ...flowRunFinished.data.flowRun, duration: 5214.5 } } }+ const request = otlpLogs.buildExportRequest({ event, environment: 'prod' })++ expect(attributeOf({ request, key: 'data.flowRun.duration' })?.value).toEqual({ doubleValue: 5214.5 })+ expect(attributeOf({ request: otlpLogs.buildExportRequest({ event: flowRunFinished, environment: 'prod' }), key: 'data.flowRun.duration' })?.value).toEqual({ intValue: '5214' })+ expect(decode(otlpLogs.encodeExportRequest(request))).toEqual(request)+ })+})++describe('otlpLogs.encodeExportRequest', () => {+ it('round-trips the built request through protobuf', () => {+ const request = otlpLogs.buildExportRequest({ event: flowRunFinished, environment: 'prod' })++ expect(decode(otlpLogs.encodeExportRequest(request))).toEqual(request)+ })++ it('keeps nanosecond timestamps and 64-bit integers above 2^53 exact', () => {+ const request = {+ resourceLogs: [{+ scopeLogs: [{+ logRecords: [{+ timeUnixNano: '1790158542318000001',+ attributes: [{ key: 'count', value: { intValue: '9007199254740993' } }, { key: 'ok', value: { boolValue: false } }],+ }],+ }],+ }],+ }++ expect(decode(otlpLogs.encodeExportRequest(request))).toEqual(request)+ })+})++function decode(bytes: Uint8Array): unknown {+ const type = Root.fromJSON(OFFICIAL_FIELD_NUMBERS).lookupType('check.ExportLogsServiceRequest')+ return type.toObject(type.decode(bytes), { longs: String, defaults: false })+}++function attributeKeysOf(request: ReturnType<typeof otlpLogs.buildExportRequest>): string[] {+ return request.resourceLogs[0].scopeLogs[0].logRecords[0].attributes.map((attribute) => attribute.key)+}++function attributeOf({ request, key }: { request: ReturnType<typeof otlpLogs.buildExportRequest>, key: string }) {+ return request.resourceLogs[0].scopeLogs[0].logRecords[0].attributes.find((attribute) => attribute.key === key)+}++const OFFICIAL_FIELD_NUMBERS = {+ nested: {+ check: {+ nested: {+ AnyValue: {+ oneofs: { value: { oneof: ['stringValue', 'boolValue', 'intValue', 'doubleValue'] } },+ fields: { stringValue: { type: 'string', id: 1 }, boolValue: { type: 'bool', id: 2 }, intValue: { type: 'int64', id: 3 }, doubleValue: { type: 'double', id: 4 } },+ },+ KeyValue: { fields: { key: { type: 'string', id: 1 }, value: { type: 'AnyValue', id: 2 } } },+ Resource: { fields: { attributes: { rule: 'repeated', type: 'KeyValue', id: 1 } } },+ InstrumentationScope: { fields: { name: { type: 'string', id: 1 } } },+ LogRecord: {+ fields: {+ timeUnixNano: { type: 'fixed64', id: 1 },+ severityNumber: { type: 'int32', id: 2 },+ severityText: { type: 'string', id: 3 },+ body: { type: 'AnyValue', id: 5 },+ attributes: { rule: 'repeated', type: 'KeyValue', id: 6 },+ observedTimeUnixNano: { type: 'fixed64', id: 11 },+ eventName: { type: 'string', id: 12 },+ },+ },+ ScopeLogs: { fields: { scope: { type: 'InstrumentationScope', id: 1 }, logRecords: { rule: 'repeated', type: 'LogRecord', id: 2 } } },+ ResourceLogs: { fields: { resource: { type: 'Resource', id: 1 }, scopeLogs: { rule: 'repeated', type: 'ScopeLogs', id: 2 } } },+ ExportLogsServiceRequest: { fields: { resourceLogs: { rule: 'repeated', type: 'ResourceLogs', id: 1 } } },+ },+ },+ },+}diff --git a/packages/server/utils/test/safe-http-post-for-status.test.ts b/packages/server/utils/test/safe-http-post-for-status.test.tsnew file mode 100644index 00000000000..8bb54e0d95e--- /dev/null+++ b/packages/server/utils/test/safe-http-post-for-status.test.ts@@ -0,0 +1,193 @@+import http from 'node:http'+import { AddressInfo } from 'node:net'+import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from 'vitest'+import { PostForStatusFailure, safeHttp } from '../src/safe-http'++const DEADLINE_MS = 300+const CHUNK = Buffer.alloc(64 * 1024, 'a')++describe('safeHttp.postForStatus', () => {+ const servers: TestServer[] = []++ beforeAll(() => {+ vi.stubEnv('AP_SSRF_ALLOW_LIST', '127.0.0.1')+ })++ afterEach(async () => {+ await Promise.all(servers.splice(0).map((server) => server.close()))+ })++ afterAll(() => {+ vi.unstubAllEnvs()+ })++ async function serve(handler: http.RequestListener): Promise<TestServer> {+ const server = await startServer(handler)+ servers.push(server)+ return server+ }++ it('sends the headers and the body and returns the status', async () => {+ const received: { headers: http.IncomingHttpHeaders, body: string }[] = []+ const server = await serve((req, res) => {+ const chunks: Buffer[] = []+ req.on('data', (chunk: Buffer) => chunks.push(chunk))+ req.on('end', () => {+ received.push({ headers: req.headers, body: Buffer.concat(chunks).toString() })+ res.writeHead(202).end('accepted')+ })+ })++ const result = await safeHttp.postForStatus({+ url: server.url,+ headers: { 'Content-Type': 'application/json', 'X-Api-Key': 'secret' },+ body: { action: 'flow.created' },+ timeoutMs: DEADLINE_MS,+ })++ expect(result).toEqual({ responded: true, status: 202 })+ expect(received[0].headers['x-api-key']).toBe('secret')+ expect(JSON.parse(received[0].body)).toEqual({ action: 'flow.created' })+ })++ it('stops reading a response body that never ends', async () => {+ const server = await serve((_req, res) => {+ res.writeHead(200)+ writeUntilClosed(res)+ })++ const startedAt = Date.now()+ const result = await safeHttp.postForStatus({ url: server.url, headers: {}, body: {}, timeoutMs: 5000 })++ expect(result).toEqual({ responded: true, status: 200 })+ expect(Date.now() - startedAt).toBeLessThan(2000)+ await expect(server.closedResponses()).resolves.toBeGreaterThanOrEqual(1)+ })++ it('returns the status when the compressed response body is broken', async () => {+ const server = await serve((_req, res) => {+ res.writeHead(200, { 'Content-Encoding': 'gzip' }).end('this is not gzip')+ })++ const result = await safeHttp.postForStatus({ url: server.url, headers: {}, body: {}, timeoutMs: DEADLINE_MS })++ expect(result).toEqual({ responded: true, status: 200 })+ })++ it('returns the status by the deadline when the body arrives one byte at a time', async () => {+ const server = await serve((_req, res) => {+ res.writeHead(200)+ const interval = setInterval(() => res.write('a'), 20)+ res.on('close', () => clearInterval(interval))+ })++ const startedAt = Date.now()+ const result = await safeHttp.postForStatus({ url: server.url, headers: {}, body: {}, timeoutMs: DEADLINE_MS })++ expect(result).toEqual({ responded: true, status: 200 })+ expect(Date.now() - startedAt).toBeLessThan(DEADLINE_MS + 1000)+ })++ it('reports a timeout when the destination never answers', async () => {+ const server = await serve(() => undefined)++ const result = await safeHttp.postForStatus({ url: server.url, headers: {}, body: {}, timeoutMs: DEADLINE_MS })++ expect(result).toMatchObject({ responded: false, failure: PostForStatusFailure.TIMEOUT })+ })++ it('reuses the connection for the next request', async () => {+ const server = await serve((_req, res) => {+ res.writeHead(200).end('ok')+ })++ await safeHttp.postForStatus({ url: server.url, headers: {}, body: {}, timeoutMs: DEADLINE_MS })+ await safeHttp.postForStatus({ url: server.url, headers: {}, body: {}, timeoutMs: DEADLINE_MS })++ expect(server.connections()).toBe(1)+ })++ it('reports a refused connection', async () => {+ const server = await startServer(() => undefined)+ await server.close()++ const result = await safeHttp.postForStatus({ url: server.url, headers: {}, body: {}, timeoutMs: DEADLINE_MS })++ expect(result).toMatchObject({ responded: false, failure: PostForStatusFailure.CONNECTION_FAILED })+ })++ it('reports a TLS failure', async () => {+ const server = await serve((_req, res) => {+ res.writeHead(200).end('ok')+ })++ const result = await safeHttp.postForStatus({ url: server.url.replace('http://', 'https://'), headers: {}, body: {}, timeoutMs: DEADLINE_MS })++ expect(result).toMatchObject({ responded: false, failure: PostForStatusFailure.TLS })+ })++ it('reports a blocked address', async () => {+ const result = await safeHttp.postForStatus({ url: 'http://10.0.0.1/', headers: {}, body: {}, timeoutMs: DEADLINE_MS })++ expect(result).toMatchObject({ responded: false, failure: PostForStatusFailure.BLOCKED })+ })+})++function writeUntilClosed(res: http.ServerResponse): void {+ if (res.destroyed) {+ return+ }+ if (res.write(CHUNK)) {+ setImmediate(() => writeUntilClosed(res))+ return+ }+ res.once('drain', () => writeUntilClosed(res))+}++function startServer(handler: http.RequestListener): Promise<TestServer> {+ const server = http.createServer(handler)+ let connectionCount = 0+ let closedResponseCount = 0+ server.on('connection', () => {+ connectionCount += 1+ })+ server.on('request', (_req, res) => {+ res.on('close', () => {+ closedResponseCount += 1+ })+ })+ return new Promise((resolve) => {+ server.listen(0, '127.0.0.1', () => {+ resolve({+ url: `http://127.0.0.1:${portOf(server.address())}/`,+ connections: () => connectionCount,+ closedResponses: () => waitFor(() => closedResponseCount),+ close: () => new Promise((resolveClose) => {+ server.closeAllConnections()+ server.close(() => resolveClose())+ }),+ })+ })+ })+}++function portOf(address: string | AddressInfo | null): number {+ if (address === null || typeof address === 'string') {+ throw new Error('The test server has no TCP port')+ }+ return address.port+}++async function waitFor(read: () => number): Promise<number> {+ for (let attempt = 0; attempt < 50 && read() === 0; attempt++) {+ await new Promise((resolve) => setTimeout(resolve, 20))+ }+ return read()+}++type TestServer = {+ url: string+ connections: () => number+ closedResponses: () => Promise<number>+ close: () => Promise<void>+}diff --git a/packages/server/worker/src/lib/execute/jobs/event-destination.ts b/packages/server/worker/src/lib/execute/jobs/event-destination.tsindex 9ae91847acb..2eaeb68238b 100644--- a/packages/server/worker/src/lib/execute/jobs/event-destination.ts+++ b/packages/server/worker/src/lib/execute/jobs/event-destination.ts@@ -1,5 +1,5 @@-import { tryCatch } from '@activepieces/core-utils'-import { safeHttp } from '@activepieces/server-utils'+import { isObject } from '@activepieces/core-utils'+import { otlpLogs, safeHttp } from '@activepieces/server-utils'import { EngineResponseStatus, EventDestinationJobData, WorkerJobType } from '@activepieces/shared'import { workerSettings } from '../../config/worker-settings'import { FireAndForgetJobResult, JobContext, JobHandler, JobResultKind } from '../types'@@ -9,27 +9,25 @@ export const eventDestinationJob: JobHandler<EventDestinationJobData, FireAndForasync execute(ctx: JobContext, data: EventDestinationJobData): Promise<FireAndForgetJobResult> {const timeoutInSeconds = workerSettings.getSettings().EVENT_DESTINATION_TIMEOUT_SECONDS- const { data: response, error } = await tryCatch(() => safeHttp.axios.request({+ const result = await safeHttp.postForStatus({url: data.webhookUrl,- method: 'POST',- headers: { 'Content-Type': 'application/json' },- data: data.payload,- timeout: timeoutInSeconds * 1000,- validateStatus: () => true,- }))+ headers: { 'Content-Type': data.contentType ?? 'application/json' },+ body: toRequestBody(data),+ timeoutMs: timeoutInSeconds * 1000,+ })- if (error !== null) {+ if (!result.responded) {ctx.log.error({webhookUrl: data.webhookUrl,- webhook: { id: data.webhookId },- error: error.message,+ webhook: { id: data.webhookId, deliveryFailure: result.failure },+ error: result.error.message,}, 'Event destination delivery failed before reaching the destination')}- else if (response.status >= MIN_FAILURE_HTTP_STATUS) {+ else if (result.status >= MIN_FAILURE_HTTP_STATUS) {ctx.log.error({webhookUrl: data.webhookUrl,webhook: { id: data.webhookId },- response: { status: response.status },+ response: { status: result.status },}, 'Event destination responded with a failure status')}@@ -37,4 +35,11 @@ export const eventDestinationJob: JobHandler<EventDestinationJobData, FireAndFor},}+function toRequestBody(data: EventDestinationJobData): unknown {+ if (data.contentType !== 'application/x-protobuf' || !isObject(data.payload)) {+ return data.payload+ }+ return Buffer.from(otlpLogs.encodeExportRequest(data.payload))+}+const MIN_FAILURE_HTTP_STATUS = 400diff --git a/packages/server/worker/test/lib/execute/jobs/event-destination.test.ts b/packages/server/worker/test/lib/execute/jobs/event-destination.test.tsnew file mode 100644index 00000000000..8524bff7e4c--- /dev/null+++ b/packages/server/worker/test/lib/execute/jobs/event-destination.test.ts@@ -0,0 +1,85 @@+import { otlpLogs, PostForStatusFailure, safeHttp } from '@activepieces/server-utils'+import { EventDestinationJobData, WorkerJobType } from '@activepieces/shared'+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'++vi.mock('../../../../src/lib/config/worker-settings', () => ({+ workerSettings: {+ getSettings: vi.fn().mockReturnValue({ EVENT_DESTINATION_TIMEOUT_SECONDS: 10 }),+ },+}))++import { eventDestinationJob } from '../../../../src/lib/execute/jobs/event-destination'+import { JobContext } from '../../../../src/lib/execute/types'++describe('eventDestinationJob', () => {+ let postSpy: ReturnType<typeof vi.spyOn<typeof safeHttp, 'postForStatus'>>++ beforeEach(() => {+ postSpy = vi.spyOn(safeHttp, 'postForStatus').mockResolvedValue({ responded: true, status: 200 })+ })++ afterEach(() => {+ postSpy.mockRestore()+ })++ it('posts a job without a content type as JSON', async () => {+ const payload = { action: 'flow.created', platformId: 'platform-1' }++ await eventDestinationJob.execute(makeContext(), makeJobData({ payload }))++ expect(postSpy).toHaveBeenCalledWith({+ url: 'https://example.com/webhook',+ headers: { 'Content-Type': 'application/json' },+ body: payload,+ timeoutMs: 10000,+ })+ })++ it('encodes the queued OTLP/JSON request to protobuf bytes for a protobuf job', async () => {+ const payload = {+ resourceLogs: [{+ scopeLogs: [{+ logRecords: [{ timeUnixNano: '1790158542318000000', eventName: 'flow.created' }],+ }],+ }],+ }++ await eventDestinationJob.execute(makeContext(), makeJobData({ payload, contentType: 'application/x-protobuf' }))++ const sent = postSpy.mock.calls[0][0]+ expect(sent.headers).toEqual({ 'Content-Type': 'application/x-protobuf' })+ expect(Buffer.isBuffer(sent.body)).toBe(true)+ expect(sent.body).toEqual(Buffer.from(otlpLogs.encodeExportRequest(payload)))+ })++ it('logs why a delivery did not reach the destination', async () => {+ postSpy.mockResolvedValue({ responded: false, failure: PostForStatusFailure.TIMEOUT, error: new Error('the destination did not answer') })+ const ctx = makeContext()++ await eventDestinationJob.execute(ctx, makeJobData({}))++ expect(ctx.log.error).toHaveBeenCalledWith(expect.objectContaining({+ webhook: { id: 'destination-1', deliveryFailure: PostForStatusFailure.TIMEOUT },+ error: 'the destination did not answer',+ }), 'Event destination delivery failed before reaching the destination')+ })+})++function makeContext(): JobContext {+ return {+ log: { info: vi.fn(), warn: vi.fn(), error: vi.fn(), debug: vi.fn() },+ } as unknown as JobContext+}++function makeJobData(overrides: Partial<EventDestinationJobData>): EventDestinationJobData {+ return {+ schemaVersion: 1,+ platformId: 'platform-1',+ webhookId: 'destination-1',+ webhookUrl: 'https://example.com/webhook',+ payload: {},+ jobType: WorkerJobType.EVENT_DESTINATION,+ ...overrides,+ }+}+diff --git a/packages/web/src/app/routes/platform/sample-data.ts b/packages/web/src/app/routes/platform/sample-data.tsindex f6328e1e2c3..c054ea3350f 100644--- a/packages/web/src/app/routes/platform/sample-data.ts+++ b/packages/web/src/app/routes/platform/sample-data.ts@@ -9,6 +9,7 @@ import {ApplicationEventName,buildMockEvent,EventDestination,+ EventDestinationFormat,EventDestinationScope,ProjectRole,SecretManagerConnectionScope,@@ -106,16 +107,27 @@ function projectRoles(): ProjectRole[] {function eventDestinations(): EventDestination[] {return [- 'https://splunk.acme.com/services/collector',- 'https://http-intake.logs.datadoghq.com/api/v2/logs',- 'https://siem.acme.com/hooks/activepieces',- ].map((url, index) => ({+ {+ url: 'https://otlp.datadoghq.com/v1/logs',+ format: EventDestinationFormat.OTLP_PROTOBUF,+ },+ {+ url: 'https://us.i.posthog.com/i/v1/logs',+ format: EventDestinationFormat.OTLP_JSON,+ },+ {+ url: 'https://siem.acme.com/hooks/activepieces',+ format: EventDestinationFormat.RAW,+ },+ ].map(({ url, format }, index) => ({id: `sample-destination-${index}`,...agedTimestamps((index + 1) * 5 * 24 * 60),platformId: SAMPLE_PLATFORM_ID,scope: EventDestinationScope.PLATFORM,url,events: SAMPLE_EVENT_NAMES.slice(0, 4),+ enabled: true,+ format,}));}diff --git a/packages/web/test/app/routes/platform/infra/event-destinations/components/event-destination-mutations.test.tsx b/packages/web/test/app/routes/platform/infra/event-destinations/components/event-destination-mutations.test.tsxindex 08314b99096..5064301fe7a 100644--- a/packages/web/test/app/routes/platform/infra/event-destinations/components/event-destination-mutations.test.tsx+++ b/packages/web/test/app/routes/platform/infra/event-destinations/components/event-destination-mutations.test.tsx@@ -4,6 +4,7 @@import {ApplicationEventName,EventDestination,+ EventDestinationFormat,EventDestinationScope,} from '@activepieces/shared';import { QueryClient, QueryClientProvider } from '@tanstack/react-query';@@ -168,6 +169,8 @@ const destination: EventDestination = {scope: EventDestinationScope.PLATFORM,events: [ApplicationEventName.FLOW_CREATED],url: 'https://old.example.com/hook',+ enabled: true,+ format: EventDestinationFormat.RAW,};const serverError = new Error('Destination not reachable');diff --git a/packages/web/test/app/routes/platform/infra/event-destinations/lib/event-destinations-collection.test.ts b/packages/web/test/app/routes/platform/infra/event-destinations/lib/event-destinations-collection.test.tsindex 233b52e18ed..2e85f58ecf3 100644--- a/packages/web/test/app/routes/platform/infra/event-destinations/lib/event-destinations-collection.test.ts+++ b/packages/web/test/app/routes/platform/infra/event-destinations/lib/event-destinations-collection.test.ts@@ -2,6 +2,7 @@import {ApplicationEventName,EventDestination,+ EventDestinationFormat,EventDestinationScope,} from '@activepieces/shared';import { beforeEach, describe, expect, it, vi } from 'vitest';@@ -37,6 +38,8 @@ const destination: EventDestination = {scope: EventDestinationScope.PLATFORM,events: [ApplicationEventName.FLOW_CREATED],url: 'https://old.example.com/hook',+ enabled: true,+ format: EventDestinationFormat.RAW,};const serverError = new Error('Request failed with status code 500');